A Collaborative Information Aggregation System for Idea Management

The Third International Conference on Internet and Web Applications and Services

A Collaborative Information Aggregation System for Idea Management Efthimios Bothos1, Dimitris Apostolou2 and Gregoris Mentzas1 1 National Technical University of Athens, Iroon Polytechniou 9 Zografou Athens Greece 157 80, 2 University of Piraeus, Karaoli & Dimitriou St. 80 Piraeus Greece 185 34 [email protected], [email protected], [email protected] productivity of communication among the actors involved without implementing a strategy for achieving shared understanding about the innovation process or rely on heavy use of modeling formalisms for representing and enacting collaboration routines and decision rationales that are time-consuming to implement and cumbersome for non-experienced users to take advantage of. The global build-up of the World Wide Web has made possible that anyone with a computer and Internet access may explore, join, and contribute to any Web community at any time. This new freedom is often attributed to the “Web 2.0 era” of services and applications that let users easily share opinions and resources [8]. Among the various technologies that leverage Web 2.0 “collective intelligence” are Information Aggregation Markets (IAMs). These are virtual stock markets whose purpose is to collect and aggregate information [9]. Participants trade on stocks that represent different outcomes of a future event; upon market closure, the price of these virtual stocks incorporates the available information with respect to that event. IAMs can be used in the innovation process for harnessing the collective intelligence of users. As reviewed in this paper however, although several tools exist for deploying IAMs, they are not fully appropriate for supporting the idea management process. First, as in most tools user participation is limited to trading, other useful mechanisms for collecting user feedback are not exploited. Second, existing tools are configured to aggregate information regarding well defined future events, which is not the case in idea management because some ideas may never be realized and the future event may never happen. The third issue is the limited information they provide as output. Besides market price, analysis of other data such as transactions and user participation may reveal useful information. These limitations urged us to develop a web–based software tool named IDeM, which is specifically designed to facilitate the use of IAMs for idea

Abstract Collaborative systems and methods are used in companies to support innovation and management of new ideas. The aggregation of innovation-related information from a community of users is a non-trivial task that requires the use of specialized collaborative systems and methods. In this paper we explore the use of Information Aggregation Markets (IAMs) for community-based idea management and present IDeM, a novel software system that is used for generating and evaluating new ideas utilizing the concept of IAMs.

1. Introduction Contemporary research on innovation stresses the importance of collaboration by means of interorganizational networks (see e.g. [1]). Moreover, the need to ensure the participation of many employees in the innovation processes of an organization has been also emphasized in the literature. It has been shown that the use of cross-functional teams with diverse occupational and intellectual backgrounds increases the likelihood of combining knowledge in novel ways [2]. In such teams, the amount and variety of information available to members is increased, enabling the creation and consequent evaluation of different ideas from a number of different perspectives [3]. Adamides and Karacapilidis [4] provide an overview of information technology systems and tools for collaboration in the innovation process. They distinguish between tools that aim to facilitate and to increase the productivity of exchange of ideas through shared workspaces [5], [6], systems for the controlled execution of routine sequences of work tasks associated to idea development projects [7] and Group Decision Support Systems that take into account explicitly the processes of idea creation, decisionmaking, negotiation and argumentation. Existing systems and tools either aim to increase the

978-0-7695-3163-2/08 $25.00 © 2008 IEEE DOI 10.1109/ICIW.2008.107

289

management. When designing it, we took into consideration that such a tool should support the generation of new ideas and should have manifold feedback collection mechanisms. Furthermore, it should be flexible enough in its configuration options while its code should be expandable in order to be adaptable to various usage scenarios according to the specific needs of different companies. At the same time its user interface should be usable without unnecessary complexity, easy to understand and more game-like. The rest of the paper proceeds as follows. Section 2 reviews the most prominent collective intelligence methods. In section 3 we describe the architecture of our tool, we present the functionalities it supports and we outline the implementation technologies we used. Furthermore we report on an experiment carried out to test IDeM. A review of related work is presented in section 4. This paper concludes with a discussion of future research directions for enhancing our tool and for further studying its use empirically.

A Gartner report [11] places IAMs as one of the emerging methods for community building within corporations. Although such a use of the method is at its very beginning and several questions have still to be answered, IAMs are characterized by their fairly easy deployment whereas on the other hand they provide added value to the organizations. The same report marks IAMs as a Web 2.0 concept; their inherent trait of leveraging small contributions from a broad community of contributors providing as output their collective intelligence, places them into this category of Web tools. The notion of Enterprise 2.0 formulated by McAfee [15] encourages the use of Web 2.0 applications including IAMs in corporate intranets as a means to increase productivity and information elicitation. The use of IAMs in the context of idea management is fairly new. To our knowledge two large scale experiments have been conducted in this direction; one in General Electric [16] and another in a large German B2B company [17]. Both experiments involve the setup of an IAM where traders not only buy and sell idea-contracts, but they also propose new ideas which are being imported in the market. The reported results suggest that IAMs are a promising method for idea management.

2. Collective intelligence with information aggregation markets When referring to collective intelligence, an underlying aggregation mechanism is implied. The aggregation mechanism elicits the collective intelligence by drawing out the pertinent information of each individual and combines it in such a way as to make it useful [10]. In the Web 2.0 era several mechanisms have emerged. The most prominent ones are content communities, collaborative tagging and folksonomies, social networks and IAMs, also known as prediction markets [11]. Content communities refer to online communities created for publishing and sharing media contents like YouTube and del.icio.us. Collaborative tagging is the practice of allowing anyone – especially consumers – to freely attach keywords or tags to content which results to the collective categorization or indexing of the underlying content [12]. Social networks in the Web 2.0 context refer to online communities which begun with Friendster and its relatives, and continued to more recent systems including MySpace and Facebook. IAMs rest on the concept of bringing a group of participants together via the Internet and allowing to trade shares of virtual stocks [13]. The efficient markets’ hypothesis states that when such a market reaches equilibrium, it encompasses all the available information [14]. This means that by observing the market prices we can obtain a fairly accurate estimation of the future.

3. The Idem Idea Management Platform In this section we present IDeM, a software system that incorporates features targeting the deployment of IAMs for idea management. The system can be applied in several usage scenarios within the idea management process, such as the following ones which are representative of typical innovation management processes. - Idea generation. The purpose of this scenario is the generation of new ideas. The game-like process of the IAMs impels users to propose their ideas which are entered in the market. Trading occurs for all ideas, both newly proposed and old ones. Participants are compensated not only according to their stock portfolio but also for their idea contributions. - Idea enhancement. The purpose of this scenario is to enhance the ideas in the market. Market participants can invest on an idea and then contribute to it by e.g. questioning some aspects of the idea, suggesting idea improvements or changes according to their personal view. - Idea evaluation. The purpose of this scenario is the evaluation of new ideas. A market is setup with a number of new ideas and traders act as evaluators. They trade idea stocks in an effort to increase the value of their stock portfolio. Transactions are used to

290

identify the most promising ideas. Explicit traders’ feedback can be requested as well. To support the aforementioned scenarios, IDeM provides functionalities for two roles: the Market Administrator and the Trader. The Administrator is responsible for setting up new markets, managing existing markets, selecting the original set of ideas included in a market and inviting traders. The corresponding organizational role is a person responsible for the innovation management process. Traders are employees who contribute with ideas or participate in the idea enhancement and evaluation.

price when many are buying (high demand) and decreases it when many are selling. A control parameter can adjust the market’s depth, which means that traders can buy more shares from the MM without causing massive price fluctuations. The original algorithm utilized a span of prices between 0 and 1 so we scaled it from 0 to 100 in order to make trading more intuitive. Moreover we configured the system so that one market contains many market makers, one for each idea-contract since in the Zocalo implementation each contract constitutes a separate market and therefore only one market maker was provided for each market. When processing a new order, the system compares the best existing offer in the book of orders to the MM’s. If the MM's price is no better than the book order, a trade with the book order is executed, otherwise the MM’s price is preferred. The imminent consequence of the MM is the increased liquidity of the market place. Depending on the market configuration, the CDA or the CDAwMM trading mechanisms may handle the transaction.

3.1. Architecture The logical architecture of our system is presented in Fig. 1. We followed a three tier architecture which allowed us to use different tools to develop our system and, furthermore, rendered it easily expandable with new features. 3.1.1. Market engine. This component comprises the core of IDeM. It is responsible for executing buy and sell transactions that traders place. Currently the Continuous Double Auction (CDA) and the CDA with Market Maker (CDAwMM) trading algorithms are implemented. In the case of CDA, the system stores active orders in a table called the “book of orders”. When a trader places an order for a particular idea-contract, the system validates the order and performs an order matching with the information stored in the book of orders. If a match does occur the correct number of shares are bought or sold and the data is updated in the database. The active orders are scanned based on their price and timestamp. Depending on the type of transaction (buy/sell) the highest or the lowest opposite offers are prioritized. In case two entries have the same price, the first entered is preferred. The system accepts limit and market buy orders. The former are placed in the form: “Buy (or Sell) xx shares at yy price”. The system will match these bid (or ask) offers at the exact yy price or lower (respectively higher) when and if opposite offers are available. When CDAwMM is in effect the book of orders is integrated with a Market Maker (MM). The system takes over this role and is always ready to accept buy and sell orders at a certain price acting as an “always there” buyer and seller. A price function inspired from the Zocalo open source tool [18] is utilized to simulate real life supply and demand conditions. The function follows a logarithmic rule that increases the MM’s

3.1.2. Market and GUI configuration. This component handles the system settings. A new market can be setup by using a guided three step process during which the system asks for a number of configuration options. A high degree of flexibility has been added and the Market Administrator can configure a number of parameters: closing date, operating hours, ideas to be assigned to the market, users to be invited. The trading mechanism, the initial endowment of money and number of contracts allocated to users can be selected as well. Furthermore the system is designed for gathering explicit input and feedback from the participants therefore two specific features that support this need can be selected: 1) whether to allow submission of new ideas from traders and if these ideas will be automatically entered in the market or the administrator should review them and add them manually 2) whether traders will be able to provide feedback with respect to existing ideas. Available types of feedback are comments and rating based on a 5 point Likert scale for aspects of the idea the administrator considers important. Moreover this component contains the market ending function that marks the market as inactive and calculates the final portfolio of participants. The final value of the contracts for the calculation of the participants’ portfolios can be determined in two ways.

291

Fig. 1. IDeM’s Architecture Either using their final price or manually setting it. In the latter case the average of the opinions of an expert committee can be used to determine the value. In general the clearing mechanism in our markets is an issue of high importance. IAMs need a well defined future event in order to provide best results. However, such a well defined future event is not applicable in IAMs for ideas since not all ideas will be realized. If the final price of the stocks is used in the payoff function, participants may trade on contracts with a high price and not based on their beliefs and the information they hold. Having selected a market ending function, the total portfolio value results from the sum of the user’s play money and the value of the stocks in her/his ownership.

fluctuation in time. Market administrators have access to additional data which are volume traded/participant/time, average idea scores and ranking of ideas. Ranking is calculated based on the idea-contracts’ Volume Weighted Average Price (VWAP) [19] value, given from the following formula: Ti

∑P Q t ,i

VWAPi =

t ,i

t =1 Ti

∑Q

(1)

t ,i

t =1

where Pt,i and Qt,i denote the price at which each trade occurred and the number of shares traded for stock i respectively. VWAP can be calculated using a user defined subset of the trading actions e.g. the last 60% of them. Using this approach, the preferences of traders are more accurately reflected since not only the price but also the volume of a trade conveys information about them.

3.1.3. Market Data interface. This component exposes market data as web services, so that other corporate applications may easily access them for further processing. The exposed data include the current ranking and portfolio value of traders and the current ranking of idea-contracts. In addition statistical data based on the information stored in IDeM’s database are computed by this component. A set of this data is accessible by traders. These are the high and low price, the price of the last transaction and the price fluctuation in time and the transactions’ volume

3.1.4. Security Module. This component controls user access (authentication function) and the pages that users have access to (authorization function). Depending on the provided credentials (user name and

292

password) the system signs users on as traders or administrators. The market administrator can setup user groups with access to a subset of active markets. This function allows several markets to run in parallel with its own group of users.

can be utilized. On the client side a standard web browser with javascript and macromedia flash support is required. Client side javascript is used to provide local data validation to reduce unnecessary interaction between the web client and the server. Flash is required for the charts that present the price and traded volume of the idea stocks with respect to time.

3.1.4. Market data handler. This component is responsible for retrieving and storing data to the underlying RDBMS. According to the role of a user the corresponding information can be read, written or deleted.

3.3. System Walkthrough We provide here a short system walkthrough aiming to demonstrate how IDeM supports the three generic scenarios outlined in section III. First the Market Administrator logs in and sets up a new market and assigns traders. S/he selects configuration options such as the trading mechanism, whether idea submissions by traders are allowed and the type of feedback traders can provide besides their market transactions. From this step onwards traders may log in and place their orders. The trading screen offers various information as depicted in Fig. 2. The high, low and last trade prices are presented in section (1) of the screen. Graphs are used for the volume and price history, see section (2). Users can view the best four orders of the market book and enter their offers in section (3). In section (4) the description of the idea

3.2. Implementation For the development of our system, we used the “Ruby On Rails” (RoR, www.rubyonrails.org) web application framework and web 2.0 technologies that allow focusing on the business logic by removing unnecessary development overhead such as multi-user support and authentication. The Model-ViewController [20] design pattern, on which RoR is based, helped us to easily code the three layered logical architecture. The IDeM data repository is based on MySql database. As far as the web server selection is concerned, the software has been tested using the popular Apache web server, nevertheless any other server with RoR support

3

2

1 4

Fig. 2. Trading screen

293

stock is given. When new idea submission by traders is allowed, a link to a relevant form is activated. The mandatory information for a new idea includes a title, an abstract and a category that is selected from a predefined list. Other data can be a description of the idea’s problem space, technical specifications, the application context and any other additional information. If feedback is activated, a hyperlink leads traders to the feedback page. Users have access to their portfolio where a list of the idea-contracts they own is presented. This is where traders see their active buy/sell offers that are not yet resolved. They have the possibility to cancel an open order. During market operation administrators can monitor the course of the market and view or add newly proposed ideas. Upon market end they can view the ranked list of the total of ideas based on the VWAP. This is the outcome of the idea evaluation by the market.

A total of 1572 trading actions were recorded in the 3 weeks time frame and a total of 34 ideas were proposed. 26 were judged as adequate and were inserted in the market as new but not all of them were traded. Through the experiment, we evaluated the IAMs effectiveness as a method that can simultaneously support generation and evaluation of new ideas. Moreover, we evaluated IDeM’s usability. The evaluation was performed with the use of questionnaires given to the participants and the expert committee. A five-point Likert scale was used to measure the opinions of the participants. The results were processed by calculating the mean (µ) and standard deviation (σ) of the responses. Participants found the game fairly interesting (µ=3,6; σ=0,8) on a 5 point Likert scale whereas most of them (68%) would be willing to take part in a similar game in the future. With respect to the perceived usefulness of IAMs in the business environment participants responded with a value of 4,2 (σ=0,6) on a 5-point scale. Regarding IDeM, the trading screens were found to be quite understandable (µ=3,6; σ=0,8) and transitions between the various screens easy enough (µ=3,8; σ=0,9). Furthermore traders agreed that the information contained in each screen was comprehensible (µ=3,8; σ=0,8) and the new idea submission procedure easy to follow. Our expert committee found the proposed ideas quite interesting (µ=4) and yielded an average of 4,5 when asked of the method’s usefulness, on the 5 point scale. Furthermore a positive answer was provided when questioned about the effectiveness of IAMs in supporting idea management. Additional indications regarding the effectiveness of the method were deduced from the fairly large number of proposed ideas (34 in total) and the fact that the 5 top-ranked ideas, based on their VWAP value, were also highly evaluated by the expert committee, following a Torrance test [21].

3.3. Evaluation We conducted an experiment in the context of a post graduate course in computer science, involving thirty one post-graduate computer science students. The subjects were not familiar with the concept of IAMs although most had a basic knowledge of how stock exchanges operate. We conducted a two-hour introductory session where the main concepts of IAMs and the functionalities of the software were explained to the subjects. Since we expected low liquidity of the market, we chose the CDAwMM trading algorithm. To alleviate possible evaluation apprehension effects, participant contribution was anonymous. The market was open continuously for 3 weeks. During that time participants could trade on ideas and propose new ones. The introduction of new ideas in the market was controlled by two appointed judges. They selected the most relevant ideas to the problem space of the experiment. The ideas that passed this test were introduced in the market as new stocks. They started at a trading price of 50 imaginary monetary units and the traders portfolios were updated with 50 shares of each one. An expert committee valuated all the idea contracts on a scale of 1 to 100 upon market closure. The mean price of the above valuations was the final value of the contracts and that price was used in the payoff function which produced the final portfolio value of each trader. The payoff function took into consideration the ideas that traders proposed. Those who proposed new ideas received a bonus of 10 extra shares of their ideas.

4. Related Work In general we have three categories of software applications implementing IAMs: 1) those that are being sold as complete solutions 2) those that are offered as online services and 3) open source software. Representatives of the first category are Hollywood Stock eXchange (www.hsx.com), InTrade (www.intrade.com) and ConsensusPoint (www.consensuspoint.com) although many more companies are about to emerge. They offer their software to companies in order to run internal IAMs. In

294

the second category we have Inklingmarkets (www.inklingmarkets.com). Users may create an account and build their markets through a web-based configuration panel. Regarding open source software, to our knowledge four tools are available: Zocalo [18], Freemarket [22], Jmarkets [23], IdeaFutures [24]. All of the open source tools are web based and their purpose is to provide a platform for experimenting with IAMs. Except for Zocalo the other software tools implement the continuous double auction mechanism. These tools are optimized for predicting future events with a well specified outcome, whereas user involvement besides trading is limited. We have to note here that IdeaFutures and Freemarket offer the possibility for traders to propose new contracts which are “judged” by the administrator who can introduce them in the market later on. IDeM’s difference to the above tools is threefold. Firstly it supports user feedback by allowing new ideacontracts submission, rating of the idea-contracts already in the system and commenting. Secondly it confronts the uncertainty of the underlying event by offering an expert based valuation of the contracts. Thirdly it provides as output not only the price of the idea-contracts but a weighted average, the VWAP, conveying more reliable information to the market administrators.

should be inserted in the market. This would mean to hold Initial Public Offerings (IPOs) and decide based on trader’s interest during them. Another solution could be to allow total anarchy in the system and publish every proposed idea in the market without any filtering. In this variant the market itself rejects low quality ideas through the ranking mechanism. An enhancement to this anarchist scenario is to use wikilike markets in which new ideas are freely modified or extended by their investors. Towards the testing and evaluation of the above issues we have planned a series of experiments among which one, testing IAMs for the evaluation of new ideas, will take place in a large multinational IT services company. In conclusion, our early results are encouraging showing that the use of IAMs is an effective and useful idea management method that promises to reduce time and cost by simultaneously supporting idea generation and evaluation in relatively short and inexpensive virtual market sessions.

10. References [1] Miles, R. E., Miles, G., Snow, C., Collaborative Entrepreneurship: How Communities of Networked Firms Use Continuous Innovation to Create Economic Wealth, Stanford University Press, Stanford, CA, 2005. [2] Nonaka, I., Takeuchi, H., The Knowledge-Creating Company, Oxford University Press, Oxford, UK, 1995. [3] Brown, S.L., Eisenhardt, K.M., “Product development: past research; present findings, and future directions”, Academy of Management Review vol.20, no. 3, pp. 343378, 1995. [4] Adamides, E. D., Karacapilidis, N. “Information Technology Support for the Knowledge and Social Processes of Innovation Management”, Technovation, vol. 26, pp. 50-59, 2006. [5] Christensen, C., Magnusson, M.G., Zetherstrom, M.B. “Implementation and Use of Collaborative Product Development Systems—Observations from Swedish Manufacturing Firms”, Department of Innovation Engineering and Management, Chalmers University of Technology, Working Paper 2003:1, Sweden, 2003. [6] Sethi, R., Pant, S., Sethi, A., “Web-based product development systems integration and new product outcomes: a conceptual framework”, J. of Product Innovation Management, vol. 20, pp. 37-56, 2003. [7] Bose, R., “Group support systems: technologies and products selection”, Industrial Management and Data Systems, vol. 103, no. 9, pp. 649-656, 2003. [8] Lin, K.-J., “Building Web 2.0”, Computer, vol. 40, pp. 101-102, 2007. [9] Chen, K. Y., Plott, C., “Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem”, Social Science Working Paper 1131, California Institute of Technology, 2002.

5. Conclusions Although the use of IAMs in the idea management process is new, existing research including our own inaugural experiment shows encouraging results but point out several issues that need to be further researched. Firstly, we confronted the issue of the large number of ideas. At the market’s closing point, 31 ideas existed. In a realistic corporate environment this number will be significantly higher. It is important for traders to be able to search, browse and navigate among them in an efficient way, such that it lets them identify those that best match their interests. To this direction, we believe that visualization of ideas can be useful. Furthermore, categorization, clustering as well personalization techniques can be utilized to help users quickly comprehend available ideas. Another issue worthy of further examination is the role of the facilitator in the idea generation scenarios. In our experiment we used a “dictatorial” method for introducing new ideas to the market i.e. appointed judges decided which ideas were to be entered. However this process may be democratized by allowing participants themselves decide on which ideas

295

[10] Watkins, J. H., “Prediction Markets as an Aggregation Mechanism for Collective Intelligence”, Lake Arrowhead Conference, 2007. [11] Gartner, “Web 2.0 Basics: Principles, Practices and Platforms for the Enterprise”, Symposium ITExpo, 2006. [12] Golder, S., Huberman, B.A., “The structure of collaborative tagging systems”, J. of the American Society for Information Science and Technology, Forthcoming. [13] Spann, M., Skiera, B., “Internet-Based Virtual Stock Markets for Business Forecasting”, Management Science, vol. 49, pp. 1310—1326, 2003. [14] Fama, E. F., “Efficient Capital Markets: A Review of Theory and Empirical Work”, J. of Finance, vol. 25, pp. 383—417, 1970. [15] McAfee, A. “Enterprise 2.0: The Dawn of Emergent Collaboration”, MIT Sloan Management Review, vol 47, pp. 21—28, 2006. [16] LaComb, C., A., Barnett, J., A., Pan, Q., “The imagination market. Inf Syst Front”, vol. 9, pp. 245– 256, 2007. [17] Soukhoroukova, A., Spann, M., Skiera, B., “Creating and Evaluating New Product Ideas with Idea Markets”.

[18] [19] [20] [21] [22] [23] [24] [25]

296

The Wharton School of the University of Pennsylvania, Marketing Department Colloquia, 2007. Zocalo, Available: http://sourceforge.net/projects/zocalo Lim, M., Coggins, R.J., “Optimal trade execution: an evolutionary approach”, The 2005 IEEE Congress on Evolutionary Computation 2, pp. 1045—1052, 2005. Model-View-Controller, Available: http://msdn2.microsoft.com/enus/library/ms978748.aspx Torrance, E.P., Torrance Tests of Creative Thinking. Scholastic Testing Service, Inc, 1974. Freemarket, Available: http://www.freemarketproject.org Jmarkets, Available: http://jmarkets.ssel.caltech.edu/ IdeaFutures, Available: http://sourceforge.net/project/showfiles.php?group_id=1 34782 Wolfers, J., Zitzewitz, E., “Prediction markets”, J. of Economic Perspective, vol. 18, pp. 107—126, 2004.