VECIMS 2003 - International Symposium on Virtual Environments, Human-Computer Interfaces, and Measurement Systems Lugano, Switzerland, 27-29 July 2003
Modelling E-Commerce Systems’ Quality with Belief Networks Antonia Stefani School of Science and Technology Hellenic Open University 16 Sachtouri Str., Patra, GR26222, Greece Phone: +30 2610 362586 Fax: +30 2610 361410 E-mail:
[email protected]
Michalis Xenos ∗ School of Science and Technology Hellenic Open University 16 Sachtouri Str., Patra, GR26222, Greece Phone: +30 2610 361485 Fax: +30 2610 362349 E-mail:
[email protected]
This paper proposes a model based on Bayesian Networks that can be used for assessing the quality of E-commerce systems, as well as defining specific quality requirements during the design process of E-commerce systems. Section 2 discusses quality issues that relate to E-commerce systems characteristics and that formulate the model’s background. In section 3 the notation and formation of Belief Networks are described, while section 4 discusses the structure and performance of the proposed model. Section 5 presents the model’s use and applicability. Finally, in section 6 conclusions and future work are discussed.
Abstract –This paper focuses on quality aspects of e-commerce systems and proposes a method for modelling such systems based on Belief Networks. The paper discusses the theoretical background of the proposed model, as well practical issues arising from its application. The basic notation and concept of Belief Networks is briefly presented, while emphasis is placed on the model’s structure and its usage. The presented model can be utilised for assessing the quality of e-commerce systems, as well as for aiding in quality assurance during the design and development phase of such systems
I. INTRODUCTION
II. THE MODEL’S THEORETICAL BACKGROUND
E-commerce is a constantly expanding field. This fact is confirmed by the increasing number of enterprises that invest into the creation of e-commerce systems and the continuous expansion of economic and commercial transactions through the Internet. E-commerce can be defined as follows [1]: sharing business information, maintaining business relationships and conducting business transactions by the means of telecommunication networks. Depending on the type of transactions performed electronically, there are two basic categories of e-commerce systems [2]: Business to Consumer (B2C) and Business-to-Business (B2B). In E-commerce systems, interaction with the end-user is conducted through web-based applications including a human-computer interface. Since all user-system communication is realized based on such interface, it is self evident that the quality of an E-commerce system is directly related to the quality of the human-computer interface through which the end-user interacts with the web-based applications. Usually, end-users value E-commerce systems that are flexible, usable, easily adaptable to their needs and that offer a full range of applications. But how can one evaluate E-commerce systems and define the extent to which they meet end-users’ requirements? To this end, it is necessary to define what constitutes a high-quality Ecommerce system as well as a methodology for evaluating the quality of E-commerce systems [3]. ∗
Dimitris Stavrinoudis Computer Engineering and Informatics Department Patras University, Rion, GR26500, Greece Phone: +30 2610 362556 Fax: +30 2610 361410 E-mail:
[email protected]
Most E-commerce systems seek to provide high quality services to the end-users, i.e. the clients, and to this end they include specific applications (modules) so as to meet specific end-user requirements. Examples of such requirements are searching capabilities, flexible navigation or the ability to group goods and applications like search engines, site maps and shopping carts have been developed in order to meet such requirements. Even if the type of applications that an Ecommerce system integrates changes in the future, the user requirements relating to the E-commerce system will remain unchanged. It is thus reasonable to conclude that the quality and evaluation methods of E-commerce systems will always be dependant on the quality of similar applications and their ability to meet end-user requirements. Such quality factors should be taken under serious consideration during the development of E-commerce systems. Past approaches about the quality of E-commerce systems are emphasizing on usability standards using techniques like feature inspection methods and collecting data about endusers’ opinion by questionnaires. These methods provide an important feedback to the researcher and their results can be utilised as a useful background for future work, however, they do not contribute directly to a dynamic model. On the contrary, the importance of the proposed model lies on its dynamic character. In the proposed model the results derived from its application are utilized for the model’s constant
Corresponding author: Michalis Xenos is IEEE Member since 1998.
0-7803-7785-0/03/$17.00 ©2003 13
improvement, thus contributing to a continuous evolvement and upgrading. The proposed model is based on the ISO 9126 quality standard [4]. Specifically, it relies on the set of those quality characteristics and sub-characteristics that are directly related to quality as perceived by the end-users. These quality characteristics are: functionality, usability, reliability and efficiency. The importance of each of the above mentioned quality characteristics depends on each E-commerce system’s specificities as well as the user requirements and developer priorities for the specific system. It should be mentioned that the development of the proposed model was mainly based on Business to Consumer (B2C) systems.
in a Node Probability Table. This table presents the probability that a ‘child’ node is assigned a certain value for each combination of possible values of the ‘parent’ nodes. For example, figure 1 presents two ‘parent’ nodes (nodes B and C) and one ‘child’ node (node A). The probability table of node A reflects the probability P(A|B,C) for all possible combinations of A, B, C. Thus, since there are two possible states for node B (b1, b2) of figure 3, three possible states for node C (c1, c2, c3) and three for node A (a1, a2, a3), then the NPT of node A will include 3*2*3=18 elements.
III. BELIEF NETWORKS
The philosophy underlying the proposed model is the creation of a dynamic network that concentrates and exploits the knowledge gained from the analysis of data gathered during previous researches and that can also use its own results for future estimations. A graphical presentation of the network is illustrated in Figure 2. The model uses nodes to represent the quality factors, characteristics and sub-characteristics of E-commerce systems. Each node is characterized by a set of possible states called evidence and is connected to its parent nodes by directed arrows. In figure 2 the central node ‘Quality’ appears in grey. This node represents the E-commerce system quality as a whole and is characterized by three possible states (evidence): ‘good’, ‘average’, and ‘poor’. The parent nodes of ‘quality’ are the nodes: ‘Functionality’, ‘Usability’, ‘Reliability’ and ‘Efficiency’, namely the quality factors that end-users value based on ISO 9126. These quality factors are marked with bold letters in the corresponding nodes of figure 2 and can also be characterized by three possible states: ‘good’, ‘average’, and ‘poor’. Each quality factor node is connected to the corresponding E-commerce systems quality characteristics, based on ISO 9126, which in turn are assigned three possible states as evidence: ‘good’, ‘average’, ‘poor’. Finally, each of these quality characteristics is connected to a number of child nodes comprising the quality sub-characteristics of E-commerce systems. The evidence in all nodes simply answers the question posed to the user whether a specific characteristic or sub-characteristic exists in the E-commerce system or not. This is a way to minimize subjectivity at this level as much as possible. The model has been developed using the Microsoft © MSBNx Authoring and Evaluation tool version 1.4.2. An example of the tool’s user interface is shown in Figure 3. Each node of the model has a Node Probability Table that presents the discrete conditional probability distribution. This table presents the relations between this node (child node) and its parent nodes. For example, the quality subcharacteristic of Learnability is represented as a child node connected to two parent nodes, as indicated by the directed arrows. Each parent node represents the relevant e-commerce characteristics, namely: ‘Easy help functions’ and ‘Correct placement of tools’. The ‘Learnability’ node has three
IV. MODEL’S DESCRIPTION
The proposed model is based on the notation and formation of Causal Probabilistic Networks, also called Belief Networks (BN) and Bayesian Networks [5, 6]. The mathematic model on which Bayesian Networks are based is the theorem developed by the mathematician and theologian Thomas Bayes. The BN are a special category of graphic models where nodes represent variables and the directed arrows the relations between them. Therefore, a BN is a graphic network that describes the relations of probabilities between the variables [7].
B
b1 b2
C
A
c1 c2 c3
a1 a2 a3
Fig. 1. A simple Bayesian Network
The use of BN not only makes it possible to define the relation between the various nodes (variables), but also to estimate consistently the way in which the initial probabilities influence uncertain conclusions, such as the quality of an ecommerce system. In this case, BN are used for future estimation, or –as also called– forward prediction. Furthermore, BN can be used to speculate about the states of the initial nodes, based on a given final and some intermediate variables. This is called backward assessment. In order to define the relations between the variables, firstly the dependent probabilities that describe the relations between a ‘child’ node and its ‘parent’ nodes must be determined for each node. If the values of each variable are distinct, then the probabilities for each node can be described
14
node. In the Node Probability Table of the quality subcharacteristic ‘Learnability’, which is presented in figure 3, the values of the probabilities vary between 0 to 1, scaling by 0.05. The probabilities of the model are based on data taken from previous studies of E-commerce systems [8].
possible states as evidence and the parent nodes have two states for evidence. The probability table for ‘Learnability’ has therefore 3*2*2 = 12 elements. One of the most important factors affecting the successful application of the model is the definition of the Node Probability Table of each Correct Place. Tools
Altern. Search Engine
Text Format
Reconnect.
Product Cataloq.
Operability
Error Recover.
FAQ
Search. Engine / page
Immed. Adording
Understa ndability Contact firm
Shop. cart / list Thank. e-mail
Access.T ext Info.
Resour. Behavior
Discount Offers
Usability
Attractiveness
Multimedia
User Feedback
Time Behavior
Maturity
Broswer Indepen dance
Help Function
Multim. Present.
Feel of Security
Reliability
Quality
Interope rability
Fault Tolerance
Recover ability
Client Profile
Dif. Server
Site Map
Undo Function
Suitability
Functio nality
Protect. Per. data
Comper. Present.
Colors
Efficiency
Direct Access
Facilit. Freq. Users
Access. Dissabil.
Check out
Cost Analysis
Learnability
Easy Help Function
Timelin. Return
Multilinqualism
Alternati ve Pres.
Security
Accuracy Search Engine Secure Payment
Shop. Cart
Trust
Shop. List
Fig. 2. Graphical presentation of the proposed model
15
Compera tive Pres.
Fig. 3. Example of a Belief Network
node of the model the available evidence (measures) related to the E-commerce system. The model can then be used to provide an estimation about the system’s quality and characterize it as ‘good’, ‘average’ and ‘poor’ also providing the corresponding probability values. It is worth mentioning that the model can provide estimations even if evidence has not been inserted in all of its nodes. Of course, more evidence inserted into the model improves the results’ accuracy. In a similar way it is possible to apply the model for obtaining results about only one of the quality factors or quality sub-characteristics. In this case, the estimations of the model can be utilized by E-commerce system developers to assess the importance of the quality sub-characteristics as well as the interaction level between parent nodes and child nodes. For instance, if the developer wants to assess the extent to which characteristics such as search engine, shopping cart, shopping list, alternative presentation methods and comparative presentation of the product affect the system’s accuracy, as shown in Figure 4, it is possible to do so by inserting various evidences for each of these characteristics. The different probability values obtained by the application of the model can assist the developer to conclude which system characteristics affect its accuracy more, and based on such conclusions to decide about the type and the number of applications to be developed.
The user can insert data (evidence) for one or more nodes. This evidence can activate the conditional probabilities of other nodes and provide an estimation using bar charts. For instance, in figure 3 evidence has been inserted for the parent nodes of ‘Learnability’. The corresponding bar chart shows that there is 54% probability that the system’s ‘Learnability’ is ‘good’ is 54%. Another advantage of the model is that it can utilize the results from its applications in order to improve the accuracy of future measurements. Namely, the results are used for the improvement of the Node Probabilities Tables thus contributing to better accuracy. V. APPLICATION OF THE MODEL The application of the model is based on the input of evidence for some nodes. If no evidence is inserted by the user, the estimations provided by the model are based on previously collected experience of the system, as this has been inserted in the Node Probability Table. New evidence affects the probabilities of the other nodes and the estimation for each node is different. This is shown clearly in the bar charts. The model can be used forwards and backwards. Forward use can be utilized to assess the overall quality of an Ecommerce system. In this case, the end-user inserts in each
16
Fig. 4. Example of forward use of the model
derived from measures and experiments conducted by the authors. After the initial application of the model and the consequent improvement of the values in the Node Probability Tables, the probability values resulting from the model’s application were in agreement with relevant studies of end-users’ opinions that were conducted using questionnaires.
On the other hand, backward use of the model provides assessments regarding the intermediate nodes, when the value of the final state of quality is defined. For example, if it is known that the accuracy of an E-commerce system is ‘good’ and all the other characteristics related to accuracy at the proposed model have low probability, the model would give the result that the probability of an accurate search engine must be at least 85%. This indicates the importance of this characteristic according to the user’s demands. Similarly, for a ‘good’ overall system quality and’good’ system usability the model’s probabilities about attractiveness would be 80% for good, 16% for average and 4% for poor. Therefore, in the backward use, inserting evidences concerning the child nodes, enables the model to provide estimations about the corresponding probability values of the parent nodes. It should be stressed that the model does not provide estimations by itself. It builds on the experience of the developer. This experience is based on three components: Ecommerce systems applications (modules), end-users’ demands and quality characteristics. The model’s estimations are based on discrete probabilities inserted into each Node Probability Table. Thus, the insertion of new evidence may change the model’s estimations. The model’s dynamic character is based on the creation of the Node Probability Tables. If the probabilities are based on accurate data that have been systematically collected, the estimation will be accurate. However, even in the case that the data of the Node Probability Tables are not completely accurate, the model can still provide results. It can learn (collect experience) and improve the results it provides. The initial values of the Node Probability Tables have been
VI. CONCLUSIONS AND FUTURE WORK This paper presented a model applied to assess the quality of E-commerce systems as far as the end-user is concerned. The model does not provide results by itself but is based on end-user’s experience and the accuracy of the evidence inserted into it. The model can be utilized as an important tool for the provision of estimations concerning the quality of Ecommerce systems under development and can therefore aid developers during the design phase. It can also be used backward for the assessment of already developed Ecommerce systems in order to identify problematic or high quality applications (modules). The proposed model is abstract enough so as to provide a general framework for assessment and estimation that can be utilized even if the form of applications comprising an E-Commerce system changes over time. Regarding future work, the authors’ goal is to provide a model with even lower level of subjectivity. This can be achieved by analysing E-commerce systems’ characteristics in a way that the user cannot provide estimation but only predefined answers. Furthermore, the nodes that correspond to the E-commerce system characteristics can be further
17
analysed into one or two levels. This is a way to improve the model’s accuracy even further. REFERENCES [1] [2] [3] [4] [5] [6]
[7] [8]
V. Ζwass, “Electronic commerce: structures and issues". International Journal of Electronic Commerce, Vol. 1, pp. 3 – 23, 1996. H. Chan, R. Lee and T. Dillon, “E-Commerce Fundamentals and Applications”. John Wiley & Sons, 2001. J. Burrows, “Information Technology standards in a changing world: the role of the users”. Computer standards & Interfaces, Vol. 20, pp. 323 – 331, 1999. ISO, “Information technology - Evaluation of software - Quality characteristics and guides for their use”. International Standard: ISO/IEC 9126, 1991. F. Jensen, “An introduction to Bayesian Networks”. UCL Press, 1996. S. Lauritzen and J. Spiegelhalter, “Local computations with probabilities on graphical structures and their applications to expert systems”. Readings in Uncertain Reasoning, San Mateo, California, pp. 415 – 448, 1990. N. Fenton, P. Krause and M. Neil, “Software Measurement Uncentairnty and Causal Modelling”. IEEE Software, pp. 116-122, July/August 2002. A. Stefani and M. Xenos, “A model for assessing the quality of ecommerce systems”. Proceedings of the Panhellenic Conference with International Participation in Human Computer Interaction (PC-HCI 2001), pp. 105- 109, 2001.
18
