Multiple ramp schemes - Information Theory, IEEE ... - IEEE Xplore

1720

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

Multiple Ramp Schemes Alfredo De Santis and Barbara Masucci Abstract—A (t; k; n; S ) ramp scheme is a protocol to distribute a secret chosen in S among a set P of n participants in such a way that: 1) sets of participants of cardinality greater than or equal to k can reconstruct the secret s; 2) sets of participants of cardinality less than or equal to t have no information on s, whereas 3) sets of participants of cardinality greater than t and less than k might have “some” information on s. In this correspondence we analyze multiple ramp schemes, which are protocols to share many secrets among a set P of participants, using different ramp schemes. In particular, we prove a tight lower bound on the size of the shares held by each participant and on the dealer’s randomness in multiple ramp schemes. s

Index Terms— Cryptography, data security, ramp schemes, randomness, secret sharing schemes.

I. INTRODUCTION A. Secret Sharing Schemes A secret sharing scheme is a technique to share a secret among a set P of participants in such a way that only qualified subsets of participants, pooling together their information, can reconstruct the secret; but subsets of participants that are not enabled to recover the secret have no information on it. Secret sharing schemes were introduced by Shamir [21] and Blakley [1]. They analyzed the case when only subsets of P of cardinality at least k, for a fixed integer k  jPj, can reconstruct the secret. These schemes are called (k; n) threshold schemes, where n = jPj: Subsequently, Ito, Saito, and Nishizeki [13] and Benaloh and Leichter [2] described a more general method of secret sharing. They showed how to realize a secret sharing scheme for any access structure, where the access structure is the family of all subsets of participants that are able to reconstruct the secret. The survey by Stinson [23] contains a unified description of results in the area of secret sharing schemes. For more information, the reader can also see the book [25]; while, for an updated bibliography on secret sharing schemes we refer the reader to [24]. The problem of establishing bounds on the size of the shares to be given to participants in secret sharing schemes is one of the basic problems in the area and has received considerable attention by several researchers. The practical relevance of this issue is based on the following observations: First, the security of any system tends to degrade as the amount of information that must be kept secret, i.e., the shares of the participants, increases. Secondly, if the shares given to participants are too long, the memory requirements for the participants will be too severe and, at the same time, the shares distribution algorithms will become inefficient. Therefore, it is important to derive significant upper and lower bounds on the information distributed to participants. The problem of estimating the amount of random bits necessary to set up the schemes has also received considerably attention. This is due to the fact that the amount of randomness needed by an Manuscript received March 5, 1997; revised May 4, 1998. The authors are with the Dipartimento di Informatica ed Applicazioni, Universit´a di Salerno, 84081 Baronissi (SA), Italy. Communicated by D. Stinson, Associate Editor for Complexity and Cryptography. Publisher Item Identifier S 0018-9448(99)04363-1.

algorithm is to be considered a computational resource, analogously to the amount of time and space needed. The quantitative study of the number of random bits needed by secret sharing schemes has been initiated in [8], where the optimality of several secret sharing schemes according to this measure has been proved. Some other result on this topic can be found in [6] and [9]. There are several situations in which more than one secret is to be shared among participants. As an example, consider the following situation, described by Simmons [22]: There is a missile battery and not all of the missiles have the same launch enable code. The problem is to devise a scheme which will allow any one, or any selected subset, of the launch enable codes to be activated in this scheme. What is needed is an algorithm such that the same pieces of private information could be used to recover different secrets. This problem could be trivially solved by realizing different secret sharing schemes, one for each of the launch enable codes, but in this case each participant should remember too much information. Another scenario in which the sharing of many secrets is important was considered by Franklin and Yung [12]. They investigated the communication complexity of unconditionally secure multiparty computation and its relations with various fault-tolerant models. They presented a general technique for parallelizing noncryptographic computation protocols, at a small cost in fault tolerance. Their technique replaces polynomial-based (single) secret sharing with a technique allowing multiple secrets to be hidden in a single polynomial. The technique applies to all of the protocols for secure computation which use polynomial-based threshold schemes and applies to all fault-tolerant models. The problem of sharing more than one secret was also considered in [4], [5], [7], [14], [16]–[18]. B. Ramp Schemes There are several practical situations in which it is not possible to give to participants all the secret information required to preserve perfect security, since they allow to achieve a certain amount of data compression at the cost of some degradation in security (see [20]). Blakley and Meadows [3] were the first authors to define schemes useful in such situations, called ramp schemes. Ramp schemes are useful in protocols for secure computation in fault-tolerant models. In fact, the protocol proposed by Franklin and Yung [12] can be viewed as a ramp scheme. Another example of ramp schemes can be found in [19] and [20]. More precisely, the authors of [20] considered the problem of sharing a secret by giving to participants a share of size strictly smaller than the size of the secret. This requirement directly implies that absolute security is not possible, that is, sets of participants not enabled to reconstruct the secret still could gain some information on it. Blundo, De Santis, and Vaccaro [7], [8] proved a tight lower bound on the size of the shares and on the dealer’s randomness in ramp schemes. In this correspondence, we formally define multiple ramp schemes, to share many secrets among a set of participants, by using the entropy approach, as done in [4] to analyze usual multisecret sharing schemes. We prove a tight lower bound on the size of the shares held by each participant and on the dealer’s randomness in multiple ramp schemes. A simple method to realize a multiple ramp scheme is the following, which we call basic multiple ramp scheme. We use many different and independent single ramp schemes, one for each of the secrets, and we distribute to each participant a share from each scheme. We show that, if each scheme in a basic multiple ramp scheme is optimal both

0018–9448/99$10.00  1999 IEEE

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

with respect to the information given to each participant and with respect to the number of random bits used, then the basic multiple ramp scheme is optimal both with respect to the information given to each participant and with respect to the number of random bits used, as well.

1721

such that 1  tj < kj  n, is a sharing of the secrets (s1 ; 1 1 1 ; s` ) 2 S1 2 1 1 1 2 S` among participants in P in such a way that, for j

= 1;

j

H (Sj A)

This correspondence is organized as follows. In Section II, we formally define multiple ramp schemes. In Section III, we present some results that will be useful to prove our limitations. In Section IV, we prove a tight lower bound on the size of the shares distributed to each participant in multiple ramp schemes. In Section V, we prove a tight lower bound on the dealer’s randomness in multiple ramp schemes. II. THE MODEL A (t; k; n; S ) ramp scheme is a protocol to distribute a secret s chosen in S among a set P of n participants in such a way that: 1) sets of participants of cardinality greater than or equal to k can reconstruct the secret s, 2) sets of participants of cardinality less than or equal to t have no information on s, whereas 3) sets of participants of cardinality greater than t and less than k might have “some” information on s: In this section we consider the case in which we want to share many secrets among a set P of n participants, using different ramp schemes. Let P = fP1 ; 1 1 1 ; Pn g be the set of participants. Let SC = S1 2 1 1 1 2 S` be the set where the secrets are chosen from and let fPrSC (s1 ; 1 1 1 ; s` )g(s ;111;s )2SC be a probability distribution on SC : Let a multiple ramp scheme for secrets in SC be fixed. For any participant P 2 P , let us denote by C (P ) the set of all possible shares given to participant P: Suppose a dealer D wants to share the secrets (s1 ; 1 1 1 ; s` ) 2 SC among the participants in P (we will assume that D 62 P ). He does this by giving each participant P 2 P a share from C (P ) chosen according to some, not necessarily uniform, probability distribution. Given a set of participants A = fPi ; 1 1 1 ; Pi g  P ; where i1 < i2 < 1 1 1 < ir ; let C (A) = C (Pi ) 2 1 1 1 2 C (Pi ): Given a set of indices R = fi1 ; 1 1 1 ; ir g  f1; 1 1 1 ; `g; where i1 < i2 1 1 1 < ir ; let SR = Si 2 1 1 1 2 Si : Any multiple ramp scheme for secrets in SC and a probability distribution fPrSC (s1 ; 1 1 1 ; s` )g(s ;111;s )2SC naturally induce probability distributions on C (A) and on SR , for any A  P and for any R  f1; 1 1 1 ; `g: Denote such probability distributions by fPrC (A) (a)ga2C (A) , and by fPrS (r)gr2S , respectively. To avoid overburdening the notation, with the same symbol A we will denote both a subset of participants and the random variable taking values on C (A) according to the probability distribution fPrC (A) (a)ga2C (A) ; whereas with SR we will denote both the subset of secrets and the random variable taking values on SR according to the probability distribution fPrS (r )gr 2S : For j = 1; 1 1 1 `; denote by H (Sj ) the entropy (for some basic properties of the entropy consult the Appendix) of fPrS (sj )gs 2S ; for any A  P , denote by H (A) the entropy of fPrC (A) (a)ga2C (A) ; and for any R  f1; 1 1 1 ; `g, denote by H (SR ) the entropy of S

g2

(r )

r

S

:

We formally define multiple ramp schemes by using the entropy function, as done in [4] to analyze usual multisecret sharing schemes, mainly because this leads to a compact and simple description of the schemes and because the entropy approach takes into account all probability distributions on the secrets. A multiple ramp scheme is defined as follows. Definition 2.1 (Multiple Ramp Scheme): A multiple ramp scheme tj ; kj ; n; Sj gj =1;111;` ); where tj and kj are positive integers

R = (f

;`

1) Any set of at least kj participants can recover sj : Formally, for all A  P with jAj  kj ; it holds that

C. Organization

fPr

111

= 0:

2) Any set of at most tj participants, even knowing an arbitrary set of secrets, has no more information on sj than that already conveyed by the known secrets. Formally, for all A  P with jAj  tj and R  f1; 1 1 1 ; `g; it holds that H (Sj jASR ) = H (Sj jSI (A) SR ), where I (A) =

f: j j  g i

A

ki

:

Throughout this correspondence we assume, without loss of generality, that t1  t2  1 1 1  t` : Notice that in this case, for any A  P such that jAj  tj , the set I (A) will be equal to

I(

A)

=

f

i < j:

j j  g  f1 1 1 1 0 1g A

ki

;

;j

:

Notice that Property 1 means that each set of values of the shares in C (A) corresponds to a unique value of the secret sj 2 Sj : In fact, H (Sj jA) = 0 is equivalent to the fact that for all a 2 C (A) with Pr (A = a) > 0 a unique sj 2 Sj exists such that Pr (Sj = sj jA = a) = 1: Moreover, Property 2 is equivalent to the state that for all a 2 C (A) and for all r 2 SR it holds that Pr (Sj =

j

sj A

=

a; SR

= r ) = Pr (Sj =

jI

sj S (A)

=

s; R

= r ):

Therefore, the probability that a secret is equal to sj , given that the shares held by A, enabling it to recover the set of secrets s, are equal to a and the subset of secrets A knows is equal to r, is the same as the probability of the secret sj , given the sets of secrets s and r: The scheme introduced by Franklin and Yung in [12] can be viewed as a multiple ramp scheme (ft; k; n; Sj gj =1;111;` ); to distribute ` secrets among n participants in such a way that: 1) any subset of at least k participants can recover all the secrets; 2) any subset of at most t participants can deduce anything about the secrets. Franklin and Yung gave a construction of such a scheme with t = k 0 `, by generalizing Shamir’s scheme [21]. Their construction is the following. Let s1 ; 1 1 1 ; s` be ` secrets each belonging to GF (2q ), where n + ` < 2q : To set up the scheme the dealer independently and uniformly chooses t = k 0 ` elements a1 ; 1 1 1 ; at in GF (2q ) and constructs the polynomial f (y )

=

s1

2

+ s2 y + s3 y +

111 +

s` y

01 + a1 y

`

`

+

111 +

at y

k

01 :

The share distributed to the ith participant is equal to f (i): It is easy to see that any k participants can interpolate their shares to recover f (y ), and hence recover all the ` secrets, whereas any t = k 0 ` participants have no information on the ` secrets. Notice that such multiple ramp scheme can be viewed as a (t; k; n; S ) ramp scheme, by considering all the ` secrets as a unique “super-secret.” In this scheme the total information given to participants is equal to nq = nq`=(k 0 t) bits, and the total number of random bits used is equal to kq: The choice of the secrets s1 ; 1 1 1 ; s` requires q` bits, while the remaining tq = tq`=(k 0 t) bits are used by the dealer to set up the scheme, that is, to choose the coefficients a1 ; 1 1 1 ; at 2 GF (2q ): Thus given the secrets, the dealer uses tq bits of randomness. A simple method to realize a multiple ramp scheme using Franklin and Yung’s construction, is the following, which we call basic multiple ramp scheme. We use ` different and independent single ramp schemes, one for each secret, and we distribute to each participant a share from each scheme. In this scheme the total

1722

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

information given to participants is equal to

n

`

H (Sj ) : k j =1 j 0 tj

Then, it holds that

The number of random bits needed by the dealer to set up the scheme, that is, the number of random bits he uses to generate the shares, is equal to

` j =1 kj

tj

0 tj H (Sj ):

Lemma 2.2: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme, and let W  P be such that jW j  tj : Then, it holds that

H (Sj 1 1 1 S` jW S1 1 1 1 Sj01 ) = H (Sj 1 1 1 S` jS1 1 1 1 Sj01 ): Proof: We have that

H (Sj 1 1 1 S` jW S1 1 1 1 Sj01 ) =

i=j

=

i=j

Proof: Fix a set C of c random variables in P and a set A a variables in P 0 C: From (15) of the Appendix we have that H (AjB )  H (AjC ): Summing up over B 2 '(b; C ) we get H (AjB )  cb H (AjC ): B 2'(b;C ) Summing up over C 2 '(c; P ) and over A 2 '(a; P 0 C ) we obtain

C 2'(c;P ) A2'(a;P0C ) B 2'(b;C )

 cb

` i=j

H (Si jS1 1 1 1 Sj01 Sj 1 1 1 Si01 W )

(1)

Lemma 3.4: Let A; B be two families of random variables and let n; ` be integers such that jAj = n and 0  `  n 0 1: Then, it holds that

n 0 1 H (AjB )  H (C jB ): ` C 2'(n0`;A)

H (Si jS1 1 1 1 Sj01 Sj 1 1 1 Si01 )

Proof: For simplicity we prove the lemma without the variable B: The proof is trivial for ` = 0, since both terms are equal to H (AjB ): The proof is by induction on `  1: Assume ` = 1 and let X 2 A be a fixed variable in A: We have that

I (W )  f1; 1 1 1 ; j 0 1g) = H (Sj 1 1 1 S` jS 1 1 1 Sj 0 ) (since

1

(from (11) of the Appendix):

H (C )

C 2'(n01;A)

Thus the lemma holds.

=

III. USEFUL LEMMAS In this section we present some results that will be useful to prove our limitations. Definition 3.1 (' Notation): Let P be a family of n random variables and let k  n be a positive integer. We denote with '(k; P ) the set of all k-element families of P : Let K = Pi 1 1 1 Pi 2 '(k; P ), where 1  i1 < 1 1 1 < ik  n: We denote with H (K ) the entropy of k distinct variables of P , that is, H (K ) = H (Pi 1 1 1 Pi ): Lemma 3.2: Let a; b; c be positive integers with a  b  let C be a family of c random variables. Then, it holds that

H (AjC ):

Thus the lemma holds.

H (Si jS1 1 1 1 Sj01 Sj 1 1 1 Si01 SI(W ) )

1

C 2'(c;P ) A2'(a;P0C )

a0b Notice that a fixed pair of sets A; B appears exactly n0 c0b times in the triple sum on the left-hand side, hence the left-hand side triple sum of (1) is equal to

(from 2) of Definition 2.1) =

H (AjB )

n0a0b H (AjB ): c0b B 2'(b;P ) A2'(a;P0B )

(from (11) of the Appendix)

`

n0a0b H (AjB ) c0b B 2'(b;P ) A2'(a;P0B ) H (AjC ):  cb C 2'(c;P ) A2'(a;P0C )

of

We prove that, if the secrets are statistically independent, these quantities are the best possible: that is, the protocol consisting of realizing different ramp schemes, one for each secret, is optimal both with respect to the size of the shares given to participants and with respect to the number of random bits used.

`

a; b; c be positive integers such that c > b and let

Lemma 3.3: Let

P be a family of n random variables such that n  maxfa; b; cg:

c and

0a H (A) = cb 0 a A2'(a;C ) H (A): B 2'(b;C ) A2'(a;B) Proof: It is easy to see that a fixed set A 2 '(a; C ) appears a exactly cb0 0a times in the double sum on the left-hand side. Thus the lemma holds.

=



X 2A

H (A 0 fX g)

X 2A0fX g X 2A0fX g +

H (A 0 fX g) + H (A 0 fX g) H (A 0 fX g)

X 2A0fX g

H (X jA 0 fX g)

(from (11) and (15) of the Appendix) =

X 2A0fX g

H (A)

(from (10) of the Appendix)

n 0 1)H (A):

=(

Therefore, the lemma is true for ` = 1: Suppose the lemma is true for ` 0 1, that is,

n 0 1 H (A)  H (D): `01 D2'(n0(`01);A)

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

Applying the lemma to a family

H (D) 

D with ` = 1 we get

1 jDj 0 1 C2' jDj0 ;D H (C ): (

1

1723

Therefore, by the inductive hypothesis and from (3), we obtain (2)

)

n 0 1 H (A) `01

 n 01 `

Therefore, by the inductive hypothesis and from (2), we obtain

n 0 1 H (A)  1 H (C ) `01 n 0 ` D2'(n0(`01);A) C 2'(n0`;D) ` H (C ) = n 0 ` C 2'(n0`;A) (from Lemma 3.2): 01 (n 0 `)=` = n01 , we have that Since n`0 1 `

Lemma 3.5: Let A; B be families of random variables and let n; ` be integers such that jAj = n and 0  `  n 0 1: Then, it holds that

n 0 1 H (AjB )  H (C jA 0 C; B ): ` C 2'(n0`;A) Proof: For simplicity we prove the lemma without the variable B: The proof is trivial for ` = 0, since both terms are equal to H (AjB ): The proof is by induction on `  1: Assume ` = 1 and let X be a fixed variable of A: We have that

=

X 2A0fX g X 2A0fX g

H (A) H (X )+

 H (A 0fX g)+

X 2A0fX g

X 2A0fX g

H (A 0 fX gjX )

H (A 0 fX gjX )

(from (16) of the Appendix)

 H (A 0fX gjX )+

X 2A0fX g

H (A 0 fX gjX )

=

X 2A

H (C jA 0 C ):

1 jDj 0 1 C2' jDj0 ;D H (C jA 0 C ): 1

I (Dj ; Sj jTj S1 1 1 1 Sj01 ) = H (Dj jTj S1 1 1 1 Sj 01 ) 0 H (Dj jTj S1 1 1 1 Sj ) = H (Sj jTj S1 1 1 1 Sj 01 ) 0 H (Sj jTj Dj S1 1 1 1 Sj 01 ): Since jTj [ Dj j = kj , from 1) of Definition 2.1 we have that H (Sj jTj Dj S1 1 1 1 Sj01 ) = 0: Moreover, from 2) of Definition 2.1

Thus the lemma holds. Lemma 4.2: Let R = (ftj ; kj ; n; Sj gj =1;2 ) be a multiple ramp scheme and let dj = kj 0 tj ; for j = 1; 2: Then, it holds that

T 2'(t ;P ) D 2'(d ;P0T

D with ` = 1 we get (

Proof: From (14) of the Appendix we have that

H (Dj jTj S1 1 1 1 Sj01 ) = H (Sj jS1 1 1 1 Sj01 ) + H (Dj jTj S1 1 1 1 Sj ):

n 0 1 H (A)  H (DjA 0 D): `01 D2'(n0(`01);A) H (DjA 0 D) 

H (Dj jTj S1 1 1 1 Sj01 ) = H (Sj jS1 1 1 1 Sj01 ) + H (Dj jTj S1 1 1 1 Sj ):

Therefore,

Therefore, the lemma is true for ` = 1: Suppose the lemma true for ` 0 1, that is,

Applying the lemma to a set

Lemma 4.1: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme, let Tj 2 '(tj ; P ), and let Dj 2 '(kj 0 tj ; P 0 Tj ); for j = 1; 1 1 1 ; `: Then, it holds that

H (Sj jTj S1 1 1 1 Sj01 ) = H (Sj jSI(T ) S1 1 1 1 Sj01 ): Since I (Tj )  f1; 1 1 1 ; j 0 1g, it holds that H (Sj jSI(T ) S1 1 1 1 Sj01 ) = H (Sj jS1 1 1 1 Sj01 ):

H (A 0fX gjX )

C 2'(n01;A)

A share is the information distributed to each participant in the scheme used to reconstruct the secret values. An important problem in the area of secret sharing schemes is to establish bounds on the size of the shares. In fact, the security of a system degrades as the amount of information that must keep secret grows. We measure the size of the shares with the logarithm of the size of the sets from which they are taken, that is, by the number of bits necessary to their representation. In this section we prove a tight lower bound on the size of the shares for multiple ramp schemes. This bound shows that the basic multiple ramp scheme, described in Section II, is optimal with respect to the information given to each participant.

we get

(from (13) of the Appendix) =

` H (C jA 0 C ) (from Lemma 3.2): n 0 ` C 2'(n0`;A) n01 (n 0 `)=` = n01 , we have that `01 ` n 0 1 H (A)  H (C jA 0 C ): ` C 2'(n0`;A)

IV. A LOWER BOUND ON THE SIZE OF THE SHARES

Thus the lemma holds.

n 0 1)H (A) =

Since

H (C jA 0 C )

Thus the lemma holds.

n 0 1 H (A)  H (C ): ` C 2'(n0`;A)

(

=

D2'(n0(`01);A) C 2'(n0`;D)

)

(3)

n 0 d1  n 0t1d 2 t2

H (D1 jT1 S1 )

)

n01 d1 0 1 H (D2 jT2 S1 ): n 0 1 T 2'(t ;P ) D 2'(d ;P0T ) d2 0 1

1724

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

Proof: For simplicity, we prove the lemma without the variable

S1 : We distinguish two cases: d1  d2 and d1 > d2 : Assume d1  d2 : From Lemma 3.4 with ` = d2 0 d1 , A = D2 , and B = T1 we have that H (D1jT1 )  dd200d1 H (D2 jT1 ): 2 1 D 2'(d ;D ) Let

Therefore, we have that

T 2'(t ;P ) D 2'(d ;P0T

d2 0 1 t2 d2 0 d1 t1  n0t 0d n0t 0d 1 1 2 1 t2 0 t1 d2 0 d1 1 H (D2 jT2 ) T 2'(t ;P ) D 2'(d ;P0T ) n 0 d1 n01 t1 d1 0 1 = n 0 d2 n01 d2 0 1 t2 1 H (D2 jT2 ): T 2'(t ;P ) D 2'(d ;P0T )

T2 2 '(t2 ; P ): Summing up over D2 2 '(d2 ; P 0 T2 ) we get

D 2'(d ;P0T

)

D 2'(d ;D

H (D1 jT1 ) )

 dd 00d1

H (D2 jT1 ):

2

2

D 2'(d ;P0T

1

)

From Lemma 3.2 with a = d1 , b = d2 , and C = P0 T2 , we have that

D 2'(d ;P0T

)

D 2'(d ;D =

H (D1 jT1 ) )

n 0 t2 0 d1 H (D1 jT1 ): d2 0 d1 D 2'(d ;P0T )

H (D1 jT1 ) )

Summing up over

T 2'(t ;T

)

d2 0 1 d2 0 d1  n0t 0d 2 1 d2 0 d1

D 2'(d ;P0T

H (D2 jT1 ):

)

1

)

D 2'(d ;P0T

T1 2 '(t1 ; P )

)

and over

D1 2 '(d1 ; P 0 T1 )

H (D1 jT1 ) )

d2 0 1 t2 d2 0 d1 t1  n0t 0d H (D2 jT2 ): 2 1 T 2'(t ;P ) D 2'(d ;P0T ) d2 0 d1

H (D1 jT1 ) )

 d 10 1 T 2' t ;P d 0d 1 H (D jD 0 D ; T ):

H (D1 jT1 )

T2 2 '(t2 ; P ) we obtain

T 2'(t ;P ) T 2'(t ;T

Summing up over we get

D 2'(d ;D

T 2'(t ;P ) D 2'(d ;P0T

d2 0 1 d2 0 d1  n0t 0d H (D2 jT1 ) 2 1 T 2'(t ;T ) D 2'(d ;P0T ) d2 0 d1 d2 0 1 t2 d2 0 d1 t1 H (D2 jT2 )  n0t 0d 2 1 D 2'(d ;P0T ) d2 0 d1 (from (15) of the Appendix, since T1  T2 ):

Summing up over

H (D2 jD1 0 D2 ; T1 ):

1

d1 0 1 d1 0 d2

` = d1 0 d2 , A = D1 ,

)

T1 2 '(t1 ; T2 ) we obtain

D 2'(d ;P0T

Assume now d1 > d2 : From Lemma 3.5 with and B = T1 , we have that

H (D1 jT1 ) 

Hence

D 2'(d ;P0T

H (D1 jT1 ) )

(

1

2

2

1

2

)

D 2'(d ;P0T ) D 2'(d ;D

)

(5)

1

Each term in the triple sum on the right-hand side is of the form and Y are disjoint sets and

H (D2 jY ), where D2

Y 2 '(t1 + d1 0 d2 ; P ): Each disjoint pair of sets D2 ; Y appears exactly t +dt 0d times in the triple sum, that is, the number of ways to choose a set T1 in Y: Hence the right-hand side triple sum of (5) is equal to

t1 + d1 0 d2 t1

Y 2'(t +d

From Lemma 3.3 with have that

0d ;P ) D 2'(d ;P0Y )

H (D2 jY ):

a = d2 , b = t1 + d1 0 d2 , and c = t2 , we

(4)

t 0d Notice that a fixed pair of sets D1 ; T1 appears exactly n0 t 0t times in the triple sum on left-hand side, hence the left-hand side triple sum of (4) is equal to

n 0 t1 0 d1 t2 0 t1

T 2'(t ;P ) D 2'(d ;P0T

H (D1 jT1 ): )

H (D2 jY ) 0d ;P ) D 2'(d ;P0Y ) t2 t1 + d1 0 d2 H (D2 jT2 ): n 0 t1 0 d1 T 2'(t ;P ) D 2'(d ;P0T ) t2 0 t1 0 d1 + d2

Y 2'(t +d



IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

Then we have that

T 2'(t ;P ) D 2'(d ;P0T

1725

From (6), since

H (D1 jT1 )

n t1

)

n01 t1

t1 + d1 0 d2 t2 t1 t1 + d1 0 d2  d 01 n 0 t1 0 d1 1 d2 0 1 t2 0 t1 0 d1 + d2 1 H (D2 jT2 ) T 2'(t ;P ) D 2'(d ;P0T ) n 0 d1 n01 t1 d1 0 1 = n 0 d2 n01 d2 0 1 t2 1 H (D2 jT2 ): T 2'(t ;P ) D 2'(d ;P0T )

Lemma 4.3: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme and let dj = kj 0 tj ; for j = 1; 1 1 1 ; `: Then, it holds that

1

2'(t ;P ) D 2'(d ;P0T 1 H (D`01jT`01S1 1 1 1 S`01):

T 2'(t ;P ) D 2'(d ;P0T

H (D` jT` S1 1 1 1 S` ):

2'(t

T



)

H (D1 jT1 ) = H (S1 ) + H (D1 jT1 S1 ):

T 2'(t ;P ) D 2'(d ;P0T

 tn

1

+

)

and over

P 2D

Since

D1 2 '(d1 ; P 0 T1 )

H (P )

n 0 t1 H (S ) 1 d1

T 2'(t ;P ) D 2'(d ;P0T

H (D1 jT1 S1 ):

(6)

)

t1 0 1 H (P ) = n 0 H (P ): d 1 01 P 2P0T D 2'(d ;P0T ) P 2D

Moreover, from Lemma 3.2 we have that

T 2'(t ;P )

H (P ) = =

T 2'(n0t ;P ) P 2T

2'(d ;P0T n 0 d`01 n01 d`01 0 1 t`01 n 0 d` n01 t` d` 0 1

T 2'(t ;P ) D 2'(d ;P0T

1

d`01

n 0 d`01 n t`01 t` n 0 d` n t`01 t`

H (P )

n01 H (P ): t1 P 2P

n01 H (P ): t1 P 2P

H (D` jT` S1 1 1 1 S`01 ): )

n01 n 0 t` d` d`01 0 1 n01 n 0 t`01 d` 0 1 d`01

=

1

d`

P 2P

`

H (Sj jS1 1 1 1 Sj01 ) dj j =1 n + n 0 t` d` tn d` `

H (P )  n

1

T 2'(t ;P ) D 2'(d ;P0T

H (D` jT` S1 1 1 1 S` ): )

Thus the lemma holds. The next theorem is an immediate consequence of the previous lemma. Theorem 4.4: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme. Then, it holds that

Therefore, the left-hand side triple sum of (6) is equal to

n 0 t1 0 1 d1 0 1

)

by the inductive hypothesis we obtain

From Lemma 3.2 we have that

P 2P0T

H (D`01 jT`01 S1 1 1 1 S`01 )

;P ) D

H (D` jT` S1 1 1 1 S`01 ) = H (S` jS1 1 1 1 S`01 ) + H (D` jT` S1 1 1 1 S` ):

H (P )  H (S1 ) + H (D1 jT1 S1 ): T1 2 '(t1 ; P )

1

)

From Lemma 4.1 we get

From (11) and (13) of the Appendix we have that

Summing up over we get

T

From Lemma 4.2 we have that

Proof: The proof is by induction on `: Assume ` = 1: Let T1 2 '(t1 ; P ) and D1 2 '(d1 ; P 0 T1 ): From Lemma 4.1 we have that

P 2D

n d1

n H (P )  n H (S1 ) + d n n 0 t1 1 P 2P d1 t d1 1 1 H (D1 jT1 S1 ): T 2'(t ;P ) D 2'(d ;P0T ) Therefore, the lemma is true for ` = 1: Now, suppose the lemma true for ` 0 1, that is, `01 H (Sj jS1 1 1 1 Sj01 ) H (P )  n dj j =1 P 2P n + n 0 t`01 d`01 t n d`01 `01

`

1

=

it follows that

Thus the lemma holds.

H (Sj jS1 1 1 1 Sj01 ) H (P )  n dj j =1 P 2P n + t` d` tn n 0 d` `

n 0 t1 d1 n 0 t1 0 1 d1 0 1

P 2P

H (P )  n

` j =1

H (Sj jS1 1 1 1 Sj01 ) : kj 0 tj

1726

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

For ` = 1 Theorem 4.4 implies 6P 2P H (P )  n=(k 0 t)H (S ), that is, the lower bound on the size of the shares in single ramp schemes proved in [7]. If the secrets are uniformly and independently chosen, that is, H (Sj jS1 1 1 1 Sj01 ) = H (Sj ) = log jSj j; for j = 1; 1 1 1 ; `; then we can bound the size of the shares distributed to participants, as stated by the next theorem. Theorem 4.5: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme. If the secrets are independent and each secret sj is uniformly chosen in Sj , for j = 1; 1 1 1 ; `; then it holds that

P

2P

log jC (P )j  n

`

log jS j : k 0t j

V. DEALER’S RANDOMNESS IN MULTIPLE RAMP SCHEMES

Lemma 5.1: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme. Then, it holds that `

H (P ) = H (PjS1 1 1 1 S ) + H (S1 1 1 1 S ) 0 H (S1 1 1 1 S jP ):

k

j

for

`

`

j = 1; 1 1 1 ; `; it follows that H (S1 1 1 1 S jP ) = 0: `

H (P ) = H (PjS1 1 1 1 S ) + H (S1 1 1 1 S ) `

and the lemma holds.

`

H (S jS1 1 1 1 S 01 ) + H (PjS 1 1 1 S W P ): 1 k 0t j

j =m

j

j

`

j

`

W  P ; the

R0 = (ft 0 jW j; k 0 jW j; n 0 jW j; S g 111 ) on P 0 W: In fact, from Definition 2.1 it follows that, for any A 2 '(t 0 jW j; P 0 W ) and for j = m; 1 1 1 ; `; it holds that H (S jAS 1 1 1 S 0 W ) = H (S jS 1 1 1 S 0 ) that is, if t 0 jW j participants in P 0 W pool together their shares, knowing the shares in W and the secrets s ; 1 1 1 ; s 0 ; have no more j

j

j =m;

j

;`

j

1

j

1

m

1

j

m

1

j

1

1

m

information on sj than that already conveyed by the known secrets. Similarly, for any A 2 '(kj 0 jW j; P 0 W ) and for j = m; 1 1 1 ; `; it holds that H (Sj jAS1 1 1 1 Sm01 W ) = 0, that is, if kj 0 jW j participants in P 0 W pool together their shares, knowing the shares in W and the secrets s1 ; 1 1 1 ; sm01 ; can reconstruct the secret sj : Analogously to Theorem 4.4, one can easily prove that there exists P 2 P 0 W such that

H (S jS1 1 1 1 S 01 S 1 1 1 S 01 W ) : k 0t

`

H (P jS1 1 1 1 S 01 W ) 

j

m

m

m

j

j =m

j

j

jW j  t 0 1 < t  1 1 1  t

From Definition 2.1, since follows that

m

m

`

it

H (S jS1 1 1 1 S 01 S 1 1 1 S 01 W ) = H (S jS1 1 1 1 S 01 S 1 1 1 S 01 SI( ) ) = H (S jS1 1 1 1 S 01 S 1 1 1 S 01 ) (since I (W )  f1; 1 1 1 ; m 0 1g): j

m

m

j

j

m

m

j

j

m

m

j

W

Therefore,

H (P jS1 1 1 1 S 01 W ) 

`

H (S jS1 1 1 1 S 01 ) : k 0t j

m

j =m

j

j

(7)

j

From (14) of the Appendix we have that

I (P ; S1 1 1 1 S jS1 1 1 1 S 01 W ) = H (P jS1 1 1 1 S 01 W ) 0 H (P jS1 1 1 1 S W ) = H (S1 1 1 1 S jS1 1 1 1 S 01 W ) 0 H (S1 1 1 1 S jS1 1 1 1 S 01 W P ): m

m

Proof: The mutual information I (P ; S1 1 1 1 S` ) can be written either as H (P ) 0 H (PjS1 1 1 1 S` ) or as H (S1 1 1 1 S` ) 0 H (S1 1 1 1 S` jP ) (see (12) of the Appendix). Hence

Since n Hence,

H (PjS1 1 1 1 S W ) 

`

H (P ) = H (PjS1 1 1 1 S ) + H (S1 1 1 1 S ):

`

Lemma 5.2: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme and let 1  m  `: Then, for any W  P such that jW j  tm 0 1, there exists P 2 P 0 W such that

Proof: Given the secrets s1 ; 1 1 1 ; sm01 and the set ramp scheme R naturally induces a ramp scheme

j

Randomness is a fundamental resource, and plays an important role in several areas of theoretical computer science, as algorithm design, complexity, and cryptography. Since truly random bits are hard to obtain, the amount of randomness used in computation is an important issue in many applications. The Shannon entropy of the random source generating the random bits represents the most general and natural measure of randomness. Indeed, Knuth and Yao [15] have shown that the entropy of a random variable X is approximatively equal to the average number of tosses of an unbiased coin necessary to simulate the outcomes of X: The quantitative study of the number of random bits needed by secret sharing schemes has been initiated in [8], where the optimality of several secret sharing schemes according to this measure has been proved. Some other result on this topic can be found in [9]. In this section we define and analyze the measure for the amount of randomness in a multiple ramp scheme. The total randomness present in a multiple ramp scheme R = (ftj ; kj ; n; Sj gj =1;111;` ) on a set P of n participants is equal to the entropy H (P ): This takes into account also the randomness H (S1 1 1 1 S` ) of the secrets. The dealer’s randomness is the randomness needed by the dealer to set up a multiple ramp scheme for secrets in SC = S1 21 1 12S` , that is, the randomness he uses to generate the shares, given that the set SC and the probability distribution fPrSC (s1 ; 1 1 1 ; s` )g(s ;111;s )2SC are known. Therefore, for the multiple ramp scheme R, the amount of randomness used by the dealer is equal to the entropy H (PjS1 1 1 1 S` ): This randomness is needed only to generate the shares distributed to participants. Extending [8, Lemma 2.7] we obtain the following result, that relates the total randomness and the dealer’s randomness.

`

In this section we prove a tight lower bound on the dealer’s randomness in multiple ramp schemes. This bound shows that the basic multiple ramp scheme, described in Section II, is optimal with respect to the number of random bits used by the dealer to set up the scheme.

`

j

j =1

A. A Lower Bound on the Dealer’s Randomness

`

m

`

m

`

From (11) of the Appendix we have that

H (S1 1 1 1 S jS1 1 1 1 S 01 W ) = H (S1 1 1 1 S 01 jS1 1 1 1 S 01 W ) + H (S 1 1 1 S jS1 1 1 1 S 01 W ) = H (S 1 1 1 S jS1 1 1 1 S 01 ) (from Lemma 2.2): m

`

m

m

m

m

m

`

`

m

Similarly, it holds that

H (S1 1 1 1 S jS1 1 1 1 S 01 W P ) = H (S 1 1 1 S jS1 1 1 1 S 01 ): `

m

m

`

m

(8)

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

Given two sets X and Y and a joint probability distribution on their cartesian product, the conditional entropy H (X jY ) is defined as

From (8) it follows that

I (P ; S1 1 1 1 S jS1 1 1 1 S 01 W ) = 0 m

`

1727

H (X jY ) = 0

and thus

y

2

2

Y x

xjy) log Pr(xjy):

y

PrY ( ) Pr(

X

H (P jS1 1 1 1 S W ) = H (P jS1 1 1 1 S 01 W ): (9) From the definition of conditional entropy it is easy to see that H (X jY )  0: Let P 0 = P 0fP g: From (11) of the Appendix and (9) we have that Given n sets X1 ; 1 1 1 ; X and a joint probability distribution on their Cartesian product, the entropy of X1 1 1 1 X satisfies H (PjS1 1 1 1 S W ) = H (P jS1 1 1 1 S W ) + H (P 0 jS1 1 1 1 S P W ) H (X1 1 1 1 X ) = H (X1 )+ H (X2 jX1 )+ 1 1 1 + H (X jX1 1 1 1 X 01 ): = H (P jS1 1 1 1 S 01 W ) + H (PjS1 1 1 1 S P W ) 0 (10) 0 H (P jP S1 1 1 1 S P W ): Making use of (7) and since H (P jP 0 S1 1 1 1 S P W ) = 0, the lemma Given n + 1 sets X1 ; 1 1 1 ; X ; Y and a joint probability distribution on their Cartesian product, the entropy of X1 1 1 1 X given Y can follows. m

`

n

`

n

`

`

m

`

n

n

n

`

n

`

n

Theorem 5.3: Let R = (ftj ; kj ; n; Sj gj =1;111;` ) be a multiple ramp scheme. Then, it holds that `

H (PjS1 1 1 1 S ) 

t H (S jS 1 1 1 S ): 1 01 k 0t j

`

j =1

j

j

j

j

W 2 '(r; P ) be = 0 and W

a set of r participants where = ;: Let Pi 2 P 0 W be a participant satisfying Lemma 5.2. Let 1  i  `: Applying ti 0 ti01 times Lemma 5.2 we obtain Proof: Let

1

r

 r  t : Let t0 `

t

H (PjS1 1 1 1 S W 01

0 H (PjS1 1 1 1 S W ) H (S jS1 1 1 1 S 01 )  k 0t = = H (S jS1 1 1 1 S 01 ) : = (t 0 t 01 ) k 0t = Summing up over i = 1; 1 1 1 ; ` we obtain H (PjS1 1 1 1 S W ) 0 H (PjS1 1 1 1 S W ) H (S jS1 1 1 1 S 01 )  (t 0 t 01 ) k 0t =1 = h

)

t

`

t

j

j

t

`

t

`

j

j

i

j

`

i

j

i

j

`

i

`

i

j

i

j

`

j

j

i

j

t H (S jS 1 1 1 S ): 1 01 k 0t j

j =1

t

`

i

j

j

j

j

H (PjS1 1 1 1 S W )  0 and W = ;, the lemma holds. For ` = 1, Theorem 5.3 implies H (PjS )  t=(k 0 t)H (S ), that is,

Since

`

t

t

the lower bound on the dealer’s randomness in single ramp schemes proved in [8]. APPENDIX INFORMATION THEORY BACKGROUND

In this Appendix we review the basic concepts of Information Theory used in our definitions and proofs. For a complete treatment of the subject the reader is advised to consult [11]. Given a probability distribution fPrX (x)gx2X on a set X , we define the entropy of X; H (X ); as

H (X ) = 0

2

x

x

H (X1 1 1 1 X jY ) = H (X1 jY ) +

x

PrX ( ) log PrX ( )

X

(all logarithms in this correspondence are to the base 2). The entropy satisfies the following property: 0  H (X )  log jX j, where H (X ) = 0 if and only if there exists x0 2 X such that PrX (x0 ) = 1; H (X ) = log jX j if and only if PrX (x) = 1=jX j, for all x 2 X:

n

n

H (X jX1 1 1 1 X 01 Y ): i

i=2

i

(11)

The mutual information I (X ; Y ) between

X and Y is defined by I (X ; Y ) = H (X ) 0 H (X jY ) = H (Y ) 0 H (Y jX ) (12)

and enjoys the following property: I (X ; Y )  0, from which one gets

H (X )  H (X jY ):

(13)

Given n + 2 sets X; Y; Z1 ; 1 1 1 ; Zn and a joint probability distribution on their Cartesian product, the conditional mutual information I (X ; Y jZ1 1 1 1 Zn ) between X and Y given Z1 ; 1 1 1 ; Zn can be written as

I (X ; Y jZ1 1 1 1 Z

n)

= =

j

t

`

=

j

j

be expressed as

H (X jZ1 1 1 1 Z ) 0 H (X jZ1 1 1 1 Z Y ) H (Y jZ1 1 1 1 Z ) 0 H (Y jZ1 1 1 1 Z X ): n

n

n

n

(14)

Since the conditional mutual information is always nonnegative we get

H (X jZ1 1 1 1 Z )  H (X jZ1 1 1 1 Z Y ): n

n

(15)

From (11) and (15) one easily gets that for any sets Y; X1 ; 1 1 1 ; Xn and a joint probability distribution on their Cartesian product it holds that n

H (X jY )  H (X1 X2 1 1 1 X jY ): i

i=1

n

(16)

ACKNOWLEDGMENT The authors wish to thank the anonymous referees for their careful reading and useful comments. REFERENCES [1] G. R. Blakley, “Safeguarding cryptographic keys,” in Proc. AFIPS 1979 Nat. Computer Conf., June 1979, pp. 313–317. [2] J. C. Benaloh and J. Leichter, “Generalized secret sharing and monotone functions,” in Advances in Cryptology—CRYPTO ’88, S. Goldwasser, Ed., Lecture Notes in Computer Science, vol. 403. Berlin, Germany: Springer-Verlag, 1990, pp. 27–35. [3] G. R. Blakley and C. Meadows, “Security of ramp schemes,” in Advances in Cryptology—CRYPTO ’84, Lecture Notes in Computer Science, vol. 196. Berlin, Germany: Springer-Verlag, 1985, pp. 242–268. [4] C. Blundo, A. De Santis, G. Di Crescenzo, A. Giorgio Gaggia, B. Masucci, and U. Vaccaro, “Secret sharing of many secrets,” Tech. Rep., Univ. of Salerno, Salerno, Italy, 1998. [5] C. Blundo, A. De Santis, G. Di Crescenzo, A. G. Gaggia, and U. Vaccaro, “Multi-secret sharing schemes,” in Advances in Cryptology—CRYPTO ’94, Lecture Notes in Computer Science, Y. Desmedt Ed., vol. 839. Berlin, Germay: Springer-Verlag, 1994, pp. 150–163.

1728

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 5, JULY 1999

[6] C. Blundo and B. Masucci, “Randomness in multi-secret sharing schemes,” Tech. Rep., Univ. of Salerno, Salerno, Italy, 1998. [7] C. Blundo, A De Santis, and U. Vaccaro, “Efficient sharing of many secrets,” in Proc. STACS ’93 (10th Symp. Theoretical Aspects of Computer Science), Lecture Notes in Computer Science, P. Enjalbert, A. Finkel, and K. W. Wagner Eds., vol. 665. Berlin, Germay: Springer-Verlag, 1993, pp. 692–703. , “Randomness in distribution protocols,” Inform. Comput., vol. [8] 131, pp. 111–139, 1996. [9] C. Blundo, A. Giorgio Gaggia, and D. R. Stinson, “On the dealer’s randomness required in secret sharing schemes,” Des. Codes Cryptogr., vol. 11, no. 2, pp. 107–122, 1997. [10] R. M. Capocelli, A. De Santis, L. Gargano, and U. Vaccaro, “On the size of shares for secret sharing schemes,” J. Cryptol., vol. 6, pp. 157–169, 1993. [11] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991. [12] M. Franklin and M. Yung, “Communication complexity of secure computation,” in Proc. 24th Annu. ACM Symp. Theory of Computing, 1992, pp. 699–710. [13] M. Ito, A. Saito, and T. Nishizeki, “Multiple assignment scheme for sharing secret,” J. Cryptol., vol. 6, pp. 15–20, 1993. [14] E. D. Karnin, J. W. Greene, and M. E. Hellman, “On secret sharing systems,” IEEE Trans. Inform. Theory, vol. IT-29, pp. 35–41, 1983. [15] D. E. Knuth and A. C. Yao, “The complexity of nonuniform random number generation,” in Algorithms and Complexity. New York: Academic, 1976, pp. 357–428. [16] W.-A. Jackson, K. M. Martin, and C. M. O’Keefe, “Multisecret threshold schemes,” in Advances in Cryptology—CRYPTO ’93, D. R. Stinson Ed., Lecture Notes in Computer Science, vol. 773. Berlin, Germay: Springer-Verlag, 1994, pp. 126–135. , “On sharing many secrets,” in Advances in Cryptology— ASI[17] ACRYPT ’94, J. Pieprzyk and R. Safavi-Naini Eds., Lecture Notes in Computer Science, vol. 917. Berlin, Germay: Springer-Verlag, 1995, pp. 42–54. [18] , “Ideal secret sharing schemes with multiple secrets,” J. Cryptol., vol. 9, pp. 233–250, 1996. [19] W.-A. Jackson and K. M. Martin, “A combinatorial interpretation of ramp schemes,” Australas. J. Combin., vol. 14, pp. 51–60, 1996. [20] R. J. McEliece and D. Sarwate, “On sharing secrets and Reed-Solomon codes,” Commun. ACM, vol. 24, no. 9, pp. 583–584, Sept. 1981. [21] A. Shamir, “How to share a secret,” Commun. ACM, vol. 22, no. 11, pp. 612–613, Nov. 1979. [22] G. J. Simmons, “An introduction to shared secret and/or shared control schemes and their applications,” in Contemporary Cryptology. Piscataway, NJ: IEEE Press, 1991, pp. 441–497. [23] D. R. Stinson, “An explication of secret sharing schemes,” Des., Codes, Cryptogr., vol. 2, pp. 357–390, 1992. [24] , Bibliography on Secret Sharing. [Online]. Available: http://cacr. math.uwaterloo.ca/˜dstinson/ssbib.html [25] , Cryptography Theory and Practice. Boca Raton, FL: CRC, 1995.

Some Inequalities Relating Different Measures of Divergence Between Two Probability Distributions Lang Withers, Jr., Member, IEEE

Abstract— This note presents new inequalities relating different divergence measures in the family of “convex likelihood-ratio expectation” measures of Csisz´ar, Ali, and Silvey, and especially in the singleparameter family of “AM-GM” divergence measures. The most promi1 , where  is the Bhattacharyya angle nent result is that 2 4 of divergence (a true distance metric), and is the symmetric crossentropy. A pair of “log 0” divergences is also introduced and related to and . the cross-entropies

 J

I

J

J

Index Terms—Bhattacharyya angle of divergence, chi-squared statistic, Cressie–Read–Anscombe statistic, directed divergence, Hellinger integrals, Kullback–Leibler cross-entropy, relative information.

I. INTRODUCTION Measures of divergence between two probability distributions are used to associate, cluster, classify, compress, and restore signals, images and patterns [1]–[6], in many applications. Many different measures of divergence have been constructed and characterized to some extent. But this profusion of measures leaves us with the questions of how they are related, and how to select the best ones to use in specific situations. As a part of the answer to the first question, this correspondence presents some new inequalities that relate well-known divergence measures. In this note we will compare two probability distributions p(x) and q (x) over a discrete event space X = fx1 ; x2 ; 1 1 1 ; xn g, with p(x); q (x)  0 for every x 2 X , and

x2X

p(x)

=

p

=1

and q = 1. The divergence measures in this note all belong to the large family of “convex likelihood-ratio expectations” introduced by Csisz´ar, Ali, and Silvey [7]–[9]. They found that the expected value of a continuous function k of the likelihood ratio r = p(x)=q (x), E (k (r )) = qk (r ) obeys a set of basic properties of a divergence measure when and only when k(r) is a convex function of r. One of these is the discriminating property, that when we refine the discrete events and the two distributions over those events, the measure of divergence between the two should not decrease. Moreover, these properties are preserved for measures of the form h(E (k(r))) when h is an increasing function of a real variable. For convenience, we will call this family of measures the CAS (Csisz´ar–Ali–Silvey) family, and refer to k(r) = k(p=q ) as the kernel function of the expectation measure. (It plays a role somewhat analogous to that of the kernel function k(x; y ) of an integral operator.) The kernel function for a divergence measure is unique only up to an added term c(r 0 1) for any real constant c, since this term does not change the expected Manuscript received July 12, 1996; revised September 17, 1998. This work was performed in part under U.S. Army Topographic Engineering Center Contract DACA76-93-C-0009, while the author was with Pacific-Sierra Research, Arlington, VA. The author is on contract at the Naval Research Laboratory, Washington, DC 20375 USA. Communicated by A. Hero, Associate Editor for Signal Processing. Publisher Item Identifier S 0018-9448(99)04364-3.

0018–9448/99$10.00  1999 IEEE