A Method for Computing the Reputation of Multimedia ... - CiteSeerX

A Method for Computing the Reputation of Multimedia Services through Selection and Composition Pradeep K. Atrey, M. Anwar Hossain and Abdulmotaleb El Saddik Multimedia Communications Research Laboratory University of Ottawa 800 King Edward, Ottawa, Ontario, Canada ABSTRACT Multimedia services are usually selected and composed for processing, analyzing and transporting multimedia data over the Internet for end-users. The selection of these services is often performed based on their reputation, which is usually computed based on the feedback provided by the users. The users’ feedback bears many problems including the low incentive for providing ratings and the bias towards positive or negative ratings. To overcome the dependency on the user’s feedback, this paper presents a method that dynamically computes the reputation of a multimedia service based on its association with other multimedia services in a composition task. The degree of association between any two services is computed by utilizing the statistics of how often they have been composed together, which is used in our method to show the evolution of reputation over a period of time. The experimental results demonstrate the utility of the proposed method. Keywords: Reputation computation, Multimedia services, Association coefficient

1. INTRODUCTION With the tremendous growth of multimedia data over the Internet, there has been an increasing demand of multimedia services, which are used for processing, analyzing and transporting multimedia data for the end-users.1, 2 Various complex multimedia tasks, which are performed over the Internet, require selection and composition of multimedia services.3 For example, let a user who is using a PC connected to a broadband network transmit his/her small video clip and a text message written in French to another user who is using a PDA and understands only English. To accomplish this task, various multimedia services such as a language translator, video transcoder, etc. available on the web, are selected and composed. These multimedia services for individual subtasks may be selected based on various criterion. The most common criteria is the reputation of the service. In general, a multimedia service with a higher reputation is preferred over a service, which has a lower reputation in the composition.4 The reputation of a multimedia service indicates how good it has been rated for that particular (sub)task based on its various characteristics. In general, the service requesters’ feedback/rating is used to compute the reputation of a service. However, the rating based method for determining the reputation of a service bears many problems including the low incentive for providing ratings and the bias towards positive ratings.5 Moreover, the ratings provided by the service users are subjective and unfair in many cases. Therefore, there is a need for a reputation computation method that can overcome the dependency on the service users’ feedback. This paper proposes a method for the dynamic computation of the reputation of a multimedia service based on its association with other multimedia services in the compositions. The degree of association (we call it “Association Coefficient”) between any two services is computed based on how often they have been composed together. Our proposed method is based on the concept of association-based reputation referral, which is illustrated in figure 1. This concept advocates for a multimedia service to start gaining some reputation by virtue of being associated with another multimedia service. Also, a “not-so-reputed” service when associated with a “well-reputed” service has more chance of improving its reputation than the opposite case. The services, having Further author information: (E-mail: {patrey, anwar, abed}@mcrlab.uottawa.ca, Telephone: +1 (613) 562 5800 ext 2175

a reputation level greater than a threshold, are called “well-reputed”, whereas others are known as “not-soreputed”. Although it is quite fuzzy to fix this threshold value, we use 0.50 as a threshold value to differentiate between well-reputed and not-so-reputed services.

1.1 Related Work The reputation-based selection and composition of services has been widely adopted by the web community.6 Similar criteria can also be applied in the context of multimedia web services. The reputation of multimedia services, or of the service providers, may be used by the multimedia service composers for various online multimedia operations. In the past, in the web service domain, various models have been proposed for the online computation of the reputation of business parties involved in electronic transactions. These can be classified into two categories. In the first category, the trading partners or the agents compute, based on the collaborative ratings, the reputation of each other, or the reputation of a common service that they use. We call the models in this category “Collaborative-feedback models”. This category includes the works7–9 etc. The second category, which we call “Independent-feedback models”, is the one in which the reputation of a service is computed based on the independent feedback provided by the user of that service. The feedback on the different service attributes are aggregated to obtain a single reputation score of a service provider. The works5, 10 fall in the Independentfeedback models category. In the Collaborative-feedback models category, Zacharia and Maes7 presented a collaborative model that computes the reputation of a user in an online marketplace based on the ratings provided by the one user to the other user. Sreenath and Singh9 presented an agent-based collaborative approach, in which, the agents (e.g. the service users) cooperate to evaluate service providers by autonomously deciding how much weight should be given to each other’s recommendations. This method is suited more in situations when the collaborating agents have similar needs in terms of service attributes. Mui et al.8 proposed a computational model that proposes a probabilistic mechanism for inference among trust, reputation and level of reciprocity in a multiagent environment (such as an electronic market). The model adopts a strategy in which a higher reputation leads to a higher trust; the trust is the reciprocate actions between two agents; and the reciprocate actions will lead to a higher reputation. In the category of Independent-feedback models, Maximilien and Singh10 computed the reputation of a web service based on its various generic attributes (e.g. service delivery time) as well as the domain-specific attributes (e.g. accuracy and cost of the service). The domain-specific attributes may matter to a specific service user, as different users might have different preferences in terms of attributes. This model for reputation computation, due to its dependence on users’ feedback, suffers from several problems such as a lack of incentives for leaving feedback, a general bias towards positive rating, possible deceptions and collusions, etc. Recently, Sherchan et al.5 proposed a fuzzy model, in which the reputation of a web service is computed by not only using the users’ feedback but also the validation of the users’ rating behavior. The model combines the two dimensions (objective and subjective) of reputation using a fuzzy approach. Though this model claims to overcome the problem of deception and collusion, it is still dependent on the users’ feedback. In contrast to the existing feedback or rating based reputation computation methods, our method overcomes the dependency on the users’ feedback. The reputation of a new multimedia service is computed based on whether this service has been used or composed with another service.

1.2 Contribution Summary The main contributions of this paper are summarized as follows: • We propose a method of computing an association coefficient between two multimedia services based on how often they have been composed together. • This association coefficient is further used for dynamically determining the reputation of a multimedia service based on its association with another multimedia service.

Reputation of “not-so-reputed” service + reputation due to association

Reputation Observer

Reputation of “well-reputed” service

Multimedia Service ‘A’ (well-reputed)

Association

Multimedia Service ‘B’ (not-so-reputed)

Figure 1. The reputation referral based on association

• The proposed method can dynamically compute the reputation of any new multimedia service (of unknown reputation) that would be used in a composition. The remainder of this paper is organized as follows. In section 2, we formulate the problem addressed in this paper. Section 3 first presents the method for computing the association coefficient, and then describes the method of reputation computation. The results are provided in section 4. Finally, section 5 concludes the paper with a discussion on future work.

2. PROBLEM FORMULATION The problem of determining the reputation of a multimedia service is formulated as follows: 1. Let us consider a pool S = {S1 , S2 , . . . , Sn } of n multimedia services available on the Internet, which can be composed to accomplish a composite multimedia task. Out of these services, there are some well-reputed services, and the others are not-so-reputed services. A multimedia service is assumed to be well-reputed if its reputation level is greater than a threshold (0.50 in our case), otherwise it is considered as not-soreputed. For initialization, the well-reputed and the not-so-reputed multimedia services can be identified using various approaches such as based on the prior information (e.g. popularity) and the users’ feedback. If the prior information or the users’ feedback is not available, the initial reputation can be assumed to be a very low value, and over a period of time, by applying our proposed method, a multimedia service can become well-reputed by virtue of being frequently associated with the other services. 2. Let Ri (t) ∈ [0, 1], 1 ≤ i ≤ n be the reputation level of a multimedia service Si at time instant t. 3. Let λi,j (t) ∈ [0, 1] be the Association Coefficient at time instant t between the multimedia services Si and Sj . The association coefficient between two multimedia services is computed based on whether they have been composed together. The details of the computational model of the association coefficient is provided in section 3.1. The reputation level Rj (t) of a multimedia service Sj at time instant t, when composed with another multimedia service Si , is computed as follows: Rj (t) = f (Ri (t − 1), Rj (t − 1), λi,j (t))

(1)

where, Ri (t − 1) and Rj (t − 1) are the reputation levels of the multimedia services Si and Sj at time instant t − 1, respectively; and λi,j (t) is the association coefficient as described earlier. The term f is a function, details of which will be presented in the subsequent section. Note that, in a similar way, the reputation Ri (t) of service Si at time instant t can be updated due to its association with the service Sj .

Table 1. Three forms of association coefficient

λi,j =

Cosine √Ti∧j

Ti ×Tj

Dica

Jaccard

2×Ti∧j Ti +Tj

Ti∧j Ti +Tj −Ti∧j

3. PROPOSED METHOD In the following two subsections, we describe the two main steps of our method: the association coefficient computation and the reputation computation model. Section 3.1 describes how the association coefficient between two multimedia services is computed. Next, in section 3.2, we describe how this association coefficient is used to evolve the reputation of multimedia services.

3.1 Association Coefficient Computation In general, the association coefficient between the two entities (multimedia services, in our case) refers to the measure of their co-occurrence. The more often they co-occur, the more the association coefficient would be between them. Although various forms of association coefficients have been used for diverse applications in the past, the most common are: Cosine measure, Dice’s coefficient, and Jaccard coefficient.11 The applications include term similarity in documents,12 texture similarity in images,13 detection of biological molecular selfassociation,14 etc. In the context of our problem, we could compute these three types of association coefficients as follows. Let the multimedia services Si and Sj be used disjointly for accomplishing a task Ti and Tj number of times, respectively. Also, let Ti∧j be the number of times both Si and Sj have been used together for accomplishing a task. Using these statistics, the three forms of association coefficients could be computed using the formula shown in Table 1. The main drawback of computing association coefficient using any of the formulas in Table 1 is that they suffer from the problem of initialization. Initially, the terms Ti , Tj and Ti∧j are considered to be zero, and with each of them becoming 1 at the occurrence of the first composition, the association coefficient is computed to be 1 with only one composition. Subsequently, it takes several compositions to stabilize them to a true value, which could result in a big overhead for all the service pairs. Therefore, instead of using the above methods, we propose a linear combination model for computing the association coefficient in order to overcome the aforementioned problem. The proposed model is expressed as: λi,j (t) = β × λcurrent + (1 − β) × λi,j (t − 1)

(2)

where, λcurrent represents the association coefficient, which is determined only for the current composition (i.e. at time instant t). The term λcurrent is taken as 1 if a composition (Si , Sj ) exists, otherwise it is taken as zero. The terms λi,j (t) and λi,j (t − 1) are the association coefficients between the multimedia services Si and Sj at the time instances t and t − 1, respectively. The current and the past association coefficients are weighted by β and 1 − β, respectively. Note that, in equation (2), in absence of any prior information, we assume λi,j (0) = ǫ (a positive infinitesimal), for 1 ≤ i, j ≤ n.

3.2 Reputation Computation The proposed model for computing the reputation of a service considers the following two factors: 1. The rate of change of the association coefficient between the two services. A higher change in the value of the association coefficient leads to higher growth of reputation, and vice versa. 2. The degree of reputation of the service with which it is associated. Due to this, a not-so-reputed service can become well-reputed sooner when it is frequently composed (i.e. association coefficient is high) with a highly reputed service, compared to when it is seldomly composed (i.e. association coefficient is low) with a less reputed service.

Keeping in view the above two factors, we propose an exponential model, which is expressed as: Rj (t) = Z(t)−1 × Rj (t − 1) × exp(α(t))

(3)

where, the terms Rj (t) and Rj (t − 1) are the reputation values of the multimedia service Sj at time instances t and t − 1, respectively. The term exp(α(t)) acts as a growth factor for the reputation of Sj at time instant t. The term Z(t) is a normalization factor to limit the reputation value within [0,1], and is given as: Z(t) = Rj (t − 1) × exp(α(t)) + (1 − Rj (t − 1)) × exp(−α(t))

(4)

Note that we have used an exponential model of growth, and other models may also be explored. The term α(t) determines the rate of growth and is given by: α(t) = Ri (t − 1) × ∆λi,j (t) × γ

(5)

where, the term Ri (t) is the reputation of a multimedia service Si at the time instant t. The term ∆λi,j (t) = λi,j (t) − λi,j (t − 1), is the change in the association coefficient between the services Si (t) and Sj (t), at the time instant t. The term γ, which is used to control the rate of growth or decay in the reputation, can hold either of two values (growth rate and decay rate) based on whether the change in the association coefficient is positive or negative. The term γ is determined as follows:  ∆λi,j (t) ≥ 0 growth rate if γ= else ∆λi,j (t) < 0 decay rate The aforementioned reputation computation model (equation (3)) has been chosen by resorting to,15 where the authors have shown that the quality of a heuristic algorithm is determined by the accuracy of the heuristic function it uses. We will show in Section 4 that the proposed heuristics-based model provides a reasonable performance. Note that, the equation (2) shows an instance of the computation of association coefficient between two multimedia services Si and Sj , and the equation (3) depicts that the reputation of a multimedia service Sj will change based on its association with another multimedia service Si . In a real scenario, several multimedia services may be composed together to accomplish a composite task. In such a case, equation (2) and equation (3) are used iteratively for all the pairs of combinations. For example, if in a composition, a multimedia service Si is composed with two other services Sj and Sk , the reputation of each of these services is evolved by considering their pair-wise association (λi,j , λi,k and λj,k ) with each other.

4. EXPERIMENTAL RESULTS To demonstrate the utility of the proposed reputation computation method, we present results in a multimedia messaging scenario, where two media services are selected and composed. The scenario is as follows. Let two users at different sites communicate via a text-based instant messenger with a webcam facility. We assume that (a) one user is using a PC connected to a Broadband Internet, (b) the other user is using a PDA device connected to a wireless network, and (c) both speak different languages. To provide customized real-time messaging with video to each user, we will need to apply - 1) transcoding services to video streams depending on the speed of the Broadband and wireless networks, 2) language translation services e.g. English-French, etc. We considered 3 service providers for obtaining these two multimedia services, as shown in Table 2. The experiments were performed in two steps. The first step consisted of a user study that was carried out by collecting the feedback from 20 users (volunteers from our school who were knowledgeable about multimedia services and their composition) to determine the initial reputation of the services. Based on this user study, the services S2 (from group 1) and S4 (from group 2) were specified as well-reputed services. Other services were presumed to have a very low (ǫ = 0.001) initial reputation. In the second step, to simulate a scenario of instant messaging, we requested another 50 knowledgeable users to select and compose two services from the list of services provided in Table 2. These users were asked to select one service from each of the two groups - language translation and video transcoding. Based on the data (50 compositions) obtained from these users,

Table 2. The media services used in our experiment

Subtask Language translation Video transcoding

Service Initial reputation S1 - Babelfish Unknown S2 - Google 0.61 (*) S3 - Langenberg Unknown S4 - MediaCoder 0.56 (*) S5 - X Video Converter Unknown S6 - mux Video Converter Unknown ‘*’ indicates well-reputed service

Table 3. Association Coefficient between all the pairs of services at the end of 50 compositions

Service S1 S2 S3 S4 S5 S6

S1 -

S2 0.0000 -

S3 0.0000 0.0000 -

S4 0.1443 0.1964 0.0000 -

S5 0.0000 0.1147 0.0000 0.0000 -

S6 0.2586 0.1026 0.1834 0.0000 0.0000 -

initially the association coefficient between each pair of services were determined using equation (2), and then these association coefficients were used to dynamically compute the reputation of all the services using equation (3). There are 6 services of two types, 3 from each
and the users have to choose one service from each
type, type. The total number of distinct pairs are 31 × 31 . Note that the overhead of maintaining the dynamics of association coefficients between different pairs of services can be reduced by computing them only when the composition occurs. Table 3 shows the association coefficients λij , 1 ≤ i, j ≤ 6, i 6= j between pairs (Si ,Sj ). We have not shown the values in the lower diagonal half of the table as they are symmetric to the upper diagonal half. It is observed that some of the pairs of services have higher association coefficients than the others. For example, the pairs (S1 , S6 ), (S2 , S4 ), (S3 , S6 ), have association coefficients 0.2586, 0.1964, 0.1834, respectively; which is higher than that of the services (S1 , S4 ), (S2 , S5 ) and (S2 , S6 ), which have 0.1443, 0.1147 and 0.1026, respectively. Note that the association coefficients shown in the figure are based on 50 compositions. The reputation of all 6 services is dynamically computed over these compositions. In parallel, we have also requested volunteers to provide ratings for these 6 services based on their past experience. The reputation computed using our association-based method is compared with the reputation computed using the feedbackbased method. This comparison for the 6 services (S1 to S6 ) is shown over 50 compositions in 6 graphs in the figures 2a-2f, respectively. In each graph, solid and dotted lines depict the association-based method and the feedback-based method, respectively. Note that the 2 services (S2 and S6 ), one from each group, are pre-assumed to be well-reputed. From figure 2, we made the following observations: • The reputation of a not-so-reputed service increases more sharply when it has a high association coefficient, e.g. in figure 2a, the reputation of S1 increased from 0.001 to 0.1990 over the 50 compositions since its association coefficient λ1,6 with the well-reputed service S6 is 0.2586 (Refer to Table 3). On the other hand, Table 4. Difference in the reputation in Mean Square Error (MSE) per composition between Association-based and Feedback-based methods

Service MSE

S1 0.0427

S2 0.0356

S3 0.0290

S4 0.0622

S5 0.0109

S6 0.0210

0.8

0.6

Reputation

Reputation

0.8 S − not−so−reputed 1

0.4 0.2 0

0

10 20 30 40 Number of compositions

0.6

2

0.2 0

50

S − well−reputed

0.4

0

10 20 30 40 Number of compositions

(a)

(b) 0.8

0.6

Reputation

Reputation

0.8 S3 − not−so−reputed

0.4 0.2 0

0

10 20 30 40 Number of compositions

0.6

0.2 0

50

S4 − not−so−reputed

0.4

0

(c)

50

0.8

0.6

Reputation

Reputation

10 20 30 40 Number of compositions

(d)

0.8 S − not−so−reputed 5

0.4 0.2 0

50

0.6

10 20 30 40 Number of compositions

(e)

50

6

0.2 0

0

S − well−reputed

0.4

0

10 20 30 40 Number of compositions

50

(f)

Figure 2. Reputation evolution of 6 services over the 50 compositions: Association-based method (solid lines) vs. Feedbackbased method

as the association coefficient between the not-so-reputed service S5 and the reputed service S2 is smaller (λ2,5 = 0.1147), there has not been a significant increase in the reputation of S5 (i.e. from 0.001 to 0.0592), as shown in figure 2e. • The reputation of a well-reputed service further increases when it has an association with another wellreputed service, e.g. the reputation of well-reputed services S2 and S6 increased from 0.6100 to 0.8500 and from 0.5600 to 0.8693, respectively, even with a smaller value (λ2,6 = 0.1026) of the association coefficient between them. • The reputation computed using the association-based method is quite comparable with the reputation computed using the feedback-based method as the difference in the reputation obtained based on these two methods has been found to be very small. Table 4 provides this difference for all six services in terms of Mean Square Error (MSE) per composition between the two methods. This justifies the choice of the heuristic model used in our method. To summarize, the results suggest that the proposed association-based method provides a mechanism to determine reputation of the multimedia services. The association-based method not only provides reputation

values comparable to what we achieve using the feedback-based method, but it also overcomes the dependency on users’ feedback.

5. CONCLUSION The proposed association-based method for the dynamic computation of reputation of the multimedia services has been found to be quite comparable with the traditional feedback-based method. Moreover, the proposed method has an additional advantage that it overcomes the dependency on the user’s feedback in computing such reputation. Although the preliminary results in the multimedia messaging scenario are encouraging, it remains to be seen how the proposed method can be used for more complex multimedia tasks.

REFERENCES 1. D. Xu and X. Jiang, “Towards an integrated multimedia service hosting overlay,” in Proceedings of the ACM International Conference on Multimedia, pp. 96–103, (New York), October 2004. 2. X. Gu and K. Nahrstedt, “Distributed multimedia service composition with statistical QoS assurances,” IEEE Transactions on Multimedia 8(1), pp. 141–151, February 2006. 3. K. Nahrstedt and W.-T. Balke, “A taxonomy for multimedia service composition,” in Proceedings of the ACM International Conference on Multimedia, pp. 88–95, (New York), October 2004. 4. X. G. Paeda, D. Melendi, M. Garca, V. Garca, R. Garca1, and E. Riesgo, “Analysis tool for a video-ondemand service based in streaming technology,” in Proceedings of the 6th IEEE International Conference on High Speed Networks and Multimedia Communications, pp. 375–384, (Estoril, Portugal), July 2003. 5. W. Sherchan, S. W. Loke, and S. Krishnaswamy, “A fuzzy model for reasoning about reputation in web services,” in Proceedings of the 21st ACM Symposium on Applied Computing, pp. 1886–1892, (Dijon, France), April 2006. 6. J. Jin and K. Nahrstedt, “On exploring performance optimizations in web service composition,” in Proceedings of the 5th ACM/IFIP/USENIX International Conference on Middleware, pp. 115–134, (Toronto, Canada), October 2004. 7. A. M. G. Zacharia and P. Maes, “Collaborative reputation mechanisms in electronic marketplaces,” in Proceedings of the International Conference on System Sciences, (Hawaii, USA), 1999. 8. L. Mui, M. Mohtashemi, and A. Halberstadt, “A computational model of trust and reputation,” in Proceedings of the International Conference on System Sciences, pp. 280–287, (Hawaii, USA), 2002. 9. R. M. Sreenath and M. P. Singh, “Agent-based service selection,” Journal on Web Semantics , 2004. 10. E. M. Maximilien and M. P. Singh, “Conceptual model of web services reputation,” ACM SIGMOD Record 31(4), pp. 36–41, October 2002. 11. J. Chen, R. Chau, and C.-H. Yeh, “Discovering parallel text from the world wide web,” in Proceedings of the Australasian Workshop on Data Mining and Web Intelligence, pp. 157–161, (Dunedin, New Zealand), 2004. 12. D. Kuropka, “Uselessness of simple co-occurrence measures for IF and IR - A linguistic point of view,” in Proceedings of the 8th International Conference on Business Information Systems, (Poznan, Poland), April 2005. 13. M. Partio, B. Crumariuc, and M. Gabbouj, “Texture similarity evaluation using ordinal co-occurrence,” in Proceedings of the International Conference on Image Processing, pp. 1537–1540, (Singapore), October 2004. 14. D. L. Lee, C. T. Mant, and R. S. Hodges, “A novel method to measure self-association of small amphipathic molecules,” The Journal of Biological Chemistry 278(25), p. 2291822927, June 2003. 15. J. B. Oommen and L. Rueda, “A formal analysis of why heuristic functions work,” The Artificial Intelligence Journal 164, pp. 1–22, 2005.