Some Inequalities Among New Divergence Measures

Some Inequalities Among New Divergence Measures

arXiv:1010.0412v1 [cs.IT] 3 Oct 2010

Inder Jeet Taneja Departamento de Matemática Universidade Federal de Santa Catarina 88.040-900 Florianópolis, SC, Brazil. e-mail: [email protected] http://www.mtm.ufsc.br/∼taneja Abstract There are three classical divergence measures exist in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber J-divergence. Burbea-Rao [1] Jensen-Shannon divegernce and Taneja [8] arithmetic-geometric mean divergence. These three measures bear an interesting relationship among each other and are based on logarithmic expressions. The divergence measures like Hellinger discrimination, symmetric χ2 −divergence, and triangular discrimination are also known in the literature and are not based on logarithmic expressions. Past years Dragomir et al. [3], Kumar and Johnson [7] and Jain and Srivastava [4] studied different kind of divergence measures. In this paper, we worked with inequalities relating these new measures with the previous know one. An idea of exponential divergence is also developed.

Key words: J-divergence; Jensen-Shannon divergence; Arithmetic-Geometric divergence; Csiszár’s f-divergence; Information inequalities; Exponential divergence. AMS Classification: 94A17; 62B10.

1

Introduction

Let Γn =

) n X P = (p1 , p2 , ..., pn ) pi > 0, pi = 1 ,

(

n > 2,

i=1

be the set of all complete finite discrete probability distributions. For all P, Q ∈ Γn , the following measures are well known in the literature on information theory and statistics: • Hellinger Discrimination n

√ 1X √ ( pi − qi )2 , h(P ||Q) = 2 i=1 1

(1)

• Triangular Discrimination ∆(P ||Q) =

n X (pi − qi )2 i=1

pi + qi

(2)

,

• Symmetric Chi-square Divergence

Ψ(P ||Q) = χ2 (P ||Q) + χ2 (Q||P ) = where 2

χ (P ||Q) = is the well-known χ2 −divergence.

n X (pi − qi )2

qi

i=1

n X (pi − qi )2 (pi + qi )

pi qi

i=1

=

n X p2 i

i=1

qi

,

(3)

− 1,

• J-Divergence (Jeffreys [5]; Kullback-Leibler [6]) J(P ||Q) =

n X i=1

pi (pi − qi ) ln( ). qi

• Jensen-Shannon Divergence (Burbea and Rao [1]) " n X # n 2qi 2pi 1 X + qi ln . pi ln I(P ||Q) = 2 i=1 pi + qi p + q i i i=1

(4)

(5)

• Arithmetic-Geometric Mean Divergence (Taneja [8]) T (P ||Q) =

n X pi + qi i=1

2

ln

pi + qi . √ 2 pi qi

(6)

After simplification, we can write J(P ||Q) = 4 [I(P ||Q) + T (P ||Q)] .

(7)

The author [8, 11] proved the following inequality among the above five symmetric divergence measures: 1 1 1 ∆(P ||Q) 6 I(P ||Q) 6 h(P ||Q) 6 J(P ||Q) 6 T (P ||Q) 6 Ψ(P ||Q). 4 8 16

2

(8)

We observe that we have an inequality among six divergence measures three of them are logarithmic and other three are non logarithmic. More studies on divergence measures can be seen in Taneja [9, 10] and Taneja and Kumar [12]. In this paper we shall study three different kind of non logarithmic divergence measures and establish an inequality among them.

2

Different Divergence Measures

In this section we shall consider different kind of measures and study their properties and inequalities among them.

2.1

First Divergence Measure

Let us consider a measure given by n X (pi − qi )4 p D (P ||Q) = , (pi qi )3 i=1 ∗

P, Q ∈ Γn

(9)

The above measure has been study by Dragomir et al. [3], where they prove that 1 1 0 6 J(P ||Q) − ∆(P ||Q) 6 D ∗ (P ||Q), 2 12

(10)

and 1 1 0 6 Ψ(P ||Q) − J(P ||Q) 6 D ∗ (P ||Q). 2 6 Here below we shall unify and improve the inequalities given in (10) and (11).

(11)

Theorem 2.1. The following inequalities hold 8 1 1 0 6 DJ∆ (P ||Q) 6 DΨJ (P ||Q) 6 DΨT (P ||Q) 6 D ∗ (P ||Q), 4 3 24 where 1 DJ∆ (P ||Q) = J(P ||Q) − ∆(P ||Q), 2 1 DΨJ (P ||Q) = Ψ(P ||Q) − J(P ||Q) 2 3

(12)

and DΨT (P ||Q) =

1 Ψ(P ||Q) − T (P ||Q). 16

The proof of the inequalities given in (12) is based on the following lemmas: Lemma 2.1. If the function f : [0, ∞) → R is convex and normalized, i.e., f (1) = 0, then the f-divergence, Cf (P ||Q) given by Cf (P ||Q) =

n X i=1

pi , qi f qi

(13)

is nonnegative and convex in the pair of probability distribution (P, Q) ∈ Γn × Γn . Lemma 2.2. Let f1 , f2 : I ⊂ R+ → R two generating mappings are normalized, i.e., f1 (1) = f2 (1) = 0 and satisfy the assumptions: (i) f1 and f2 are twice differentiable on (a, b); (ii) there exists the real constants m, Msuch that m < Mand m6

f1′′ (x) 6 M, f2′′ (x)

f2′′ (x) > 0,

∀x ∈ (a, b),

then we have the inequalities: m Cf2 (P ||Q) 6 Cf1 (P ||Q) 6 M Cf2 (P ||Q).

(14)

The measure (13) is the well-known Csiszár’s f-divergence. The Lemma 2.1 is due to Csiszár [2] and the Lemma 2.2 is due to author [10]. Proposition 2.1. The following inequalities hold: 1 0 6 DJ∆ (P ||Q) 6 DΨJ (P ||Q). 4

(15)

Proof. We can write n

where

X 1 qi fJ∆ DJ∆ (P ||Q) = J(P ||Q) − ∆(P ||Q) = 2 i=1

pi , qi

1 1 (x − 1)2 fJ∆ (x) = fJ (x) − f∆ (x) = (x − 1) ln x − , 2 2 x+1 This gives

4

x > 0.

(16)

(17)

′ fJ∆ (x) =

and ′′ fJ∆ (x)

(x − 1)(x + 3) 1 1 − x−1 + ln x − , 2 (x + 1)2

x>0

x+1 8 (x − 1)2 (x2 + 6x + 1) = − = > 0, 2x2 (x + 1)3 2x2 (x + 1)3

∀x > 0.

(18)

′′ Thus we have fJ∆ (x) > 0 for all x > 0, and hence, fJ∆ (x) is convex for all x > 0. Also, we have fJ∆ (1) = 0. In view of this we can say that the measure DJ∆ (P ||Q) given by (13) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn × Γn .

Again, we can write n

where

X 1 DΨJ (P ||Q) = Ψ(P ||Q) − J(P ||Q) = qi fΨJ 2 i=1

pi , qi

1 (x − 1)2 (x + 1) fΨJ (x) = fΨ (x) − fJ (x) = − (x − 1) ln x, 2 2x This gives ′ fΨJ (x) =

and ′′ fΨJ (x) =

(x − 1)(2x2 + x + 1) −1 − 1 − x + ln x , 2x2 x3 + 1 x + 1 (x − 1)2 (x + 1) − = > 0, x3 x2 x3

(19)

x > 0.

(20)

x>0

∀x > 0.

(21)

′′ Thus we have fΨJ (x) > 0 for all x > 0, and hence, fΨJ (x) is strictly convex for all x > 0. Also, we have fΨJ (1) = 0. In view of this we can say that the measure DΨJ (P ||Q) given by (19) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn ×Γn .

Let us consider ′′ x(x2 + 6x + 1) fJ∆ (x) gJ∆ ΨJ (x) = ′′ = , fΨJ (x) (x + 1)4

x ∈ (0, ∞),

′′ ′′ where fJ∆ (x) and fΨJ (x) are as given by (18) and (21) respectively.

Calculating the first order derivative of the function gJ∆ ΨJ (x) with respect to x, one gets (x − 1)(x2 + 10x + 1) > 0, x < 1 ′ . (22) gJ∆ ΨJ (x) = − 5 < 0, x > 1 2(x + 1) 5

In view of (22) we conclude that the function gJ∆ ΨJ (x) is increasing in x ∈ (0, 1) and decreasing in x ∈ (1, ∞), and hence 1 M = sup gJ∆ ΨJ (x) = gJ∆ ΨJ (1) = . 4 x∈(0,∞)

(23)

By the application of (14) with (23) we get (15). Proposition 2.2. The following inequalities hold: DΨJ (P ||Q) 6

32 DΨT (P ||Q). 3

(24)


X 1 qi fΨT DΨT (P ||Q) = Ψ(P ||Q) − T (P ||Q) = 16 i=1

where

1 (x − 1)2 (x + 1) fΨT (x) = fΨ (x) − fT (x) = − 16 16x

x+1 2

ln

pi , qi

x+1 √ 2 x

(25)

, x > 0.

(26)

This gives ′ fΨT (x)

and

(x − 1)(2x2 + x + 1) x+1 1 √ = 2x ln +x−1 , − 16x2 4x 2 x

′′ fΨT (x) =

x3 + 1 x2 + 1 (x − 1)2 (x2 + x + 1) − = > 0, 16x3 4x2 (x + 1) 8x3 (x + 1)

x>0

∀x > 0.

(27)

′′ Thus we have fΨT (x) > 0 for all x > 0, and hence, fΨT (x) is convex for all x > 0. Also, we have fΨT (1) = 0. In view of this we can say that the measure DΨT (P ||Q) given by (25) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn × Γn .

Let us consider gΨJ

′′ 8(x + 1)2 fΨJ (x) = 2 , ΨT (x) = ′′ fΨT (x) x +x+1

x > 0,

′′ ′′ where fΨJ (x) and fΨT (x) are as given by (21) and (27) respectively.

Calculating the first order derivative of the function gJ∆ ΨJ (x) with respect to x, one gets

6

′ gΨJ ΨT (x)

8(x − 1)(x + 1) =− (x2 + x + 1)2

In view of (28) we conclude that the function gΨJ decreasing in x ∈ (1, ∞), and hence M = sup gΨJ

ΨT (x)

= gΨJ

> 0, x < 1 . < 0, x > 1

ΨT (x)

ΨT (1)

x∈(0,∞)

(28)

is increasing in x ∈ (0, 1) and =

32 . 3

(29)

By the application of (14) with (29) we get (24). Proposition 2.3. The following inequalities hold: DΨT (P ||Q) 6

1 ∗ D (P ||Q). 64

(30)

Proof. We can write ∗

D (P ||Q) = where fD∗ (x) =

n X i=1

pi qi fD∗ , qi

(x − 1)4 , x3/2

(31)

x > 0.

(32)

This gives fD′ ∗ (x) =

(x − 1)3 (5x + 3) , 2x5/2

x ∈ (0, ∞),

and fD′′ ∗ (x) =

3(x − 1)2 (5x2 + 6x + 5) > 0, 4x7/2

∀x > 0.

(33)

Thus we have fD′′ ∗ (x) > 0 for all x > 0, and hence, fD∗ (x) is convex for all x > 0. Also, we have fD∗ (1) = 0. In view of this we can say that the measure D ∗ (P ||Q) given by (9) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn × Γn . Let us consider gΨT

√ ′′ fΨT (x) (x2 + x + 1) x = , D ∗ (x) = fD′′ ∗ (x) 6(5x2 + 6x + 5)(x + 1)

x > 0,

′′ where fΨT (x) and fD′′ ∗ (x) are as given by (27) and (33) respectively.

7

Calculating the first order derivative of the function gΨT gets ′ gΨT D ∗ (x)

D ∗ (x)

(x − 1)(5x4 + 9x3 + 12x2 + 9x + 5) √ =− 12(5x2 + 6x + 5)2 (x + 1) x D ∗ (x)

In view of (34) we conclude that the function gΨT decreasing in x ∈ (1, ∞), and hence M = sup gΨT

D ∗ (x)

= gΨT

D ∗ (1)

with respect to x, one

> 0, x < 1 . < 0, x > 1

(34)

is increasing in x ∈ (0, 1) and =

x∈(0,∞)

1 . 64

(35)

By the application of (14) with (35) we get (30). Thus in view of Propositions 2.1 to 2.3 we get the proof of the Theorem 2.1

2.2

Second Divergence Measure

Expression (32) after simplification can be written as (x − 1)4 (x2 − 1)2 4(x − 1)2 f (x) = = − . x3/2 x3/2 x1/2 In view of expression (31), we have D∗

∗

D (P ||Q) = where

n X i=1

pi = 2ΨM (P ||Q) − 4K0 (P ||Q), qi fD∗ qi

(36)

(37)

n

and

1 X (p2i − qi2 )2 p , ΨM (P ||Q) = 2 i=1 (pi qi )3 n X (pi − qi )2 K0 (P ||Q) = . √ pi qi i=1

(38)

(39)

Measure (38) has been studied by Kumar and Johnshon [7] where they proved that Ψ(P ||Q) 6 ΨM (P ||Q).

(40)

1 1 T (P ||Q) 6 K0 (P ||Q) 6 Ψ(P ||Q). 8 16

(41)

The measure (39) has been studied by Jain and Srivastava [4] where they proved that

8

Thus combining the inequalities given in (40) and (41), we have 1 1 1 T (P ||Q) 6 K0 (P ||Q) 6 Ψ(P ||Q) 6 ΨM (P ||Q). 8 16 16 Also as a consequence of (37), we have

(42)

1 ∗ D (P ||Q) 6 ΨM (P ||Q). (43) 2 The inequalities given in (43) give a relationship among the first and second divergence measure. Let us consider below the third divergence measure

2.3

Third Divergence Measure

Let us consider the following new measure FN (P ||Q) =

√ √ n X ( pi − qi )4 i=1

pi + qi

(44)

.

Let us prove the convexity of the measure (44). We can write FN (P ||Q) = where

√ √ n X ( pi − qi )4 i=1

pi + qi

=

n X i=1

pi , qi fFN qi

√ ( x − 1)4 fFN (x) = x+1 This gives fF′ N (x)

√ √ ( x − 1)3 (x + x + 2) √ = , (x + 1)2 x

x>0

and fF′′N (x)

√ ( x − 1)2 (2x5/2 + 2x3/2 + x3 + 6x2 + x) > 0, = (x + 1)3 x5/2

∀x > 0

(45)

Thus we have fF′′N (x) > 0 for all x > 0, and hence, fFN (x) is convex for all x > 0. Also, we have fFN (1) = 0. In view of this we can say that the measure FN (P ||Q) given by (44) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn × Γn .

9

After simplification we can write √ √ 4 X n n n X pi − qi √ √ 2 1 X (pi − qi )2 = ( pi − qi ) − p + q 2 i=1 pi + qi i i i=1 i=1 √ √ 2 n n X pi − qi 1 X (pi − qi )2 =2 − 2 2 i=1 pi + qi # "i=1n √ √ 2 n X pi − qi 1 X (pi − qi )2 − =2 2 4 i=1 pi + qi i=1 1 = 2 h(P ||Q) − ∆(P ||Q) . 4 Thus we have 1 FN (P ||Q) = 2 h(P ||Q) − ∆(P ||Q) = 2Dh∆ (P ||Q). 4 Now we shall establish an inequality among the measures given by (9) , (38 and (44) . In view of (8), we have 1 h(P ||Q) 6 J(P ||Q). 8 This gives 1 1 1 h(P ||Q) − ∆(P ||Q) 6 J(P ||Q) − ∆(P ||Q), 4 8 4 i.e., 1 2 FN (P ||Q) = Dh∆ (P ||Q) 6 DJ∆ (P ||Q). 4

(46)

Applying the Theorem 2.1, we have 2 FN (P ||Q) 6

1 ∗ D (P ||Q) 96

(47)

Combining (43) and (47) we have FN (P ||Q) 6

1 1 ∗ D (P ||Q) 6 ΨM (P ||Q). 192 96

(48)

Thus the above inequality (48) establishes a relationship with these three measures presented in this section.

10

2.4

Forth Divergence Measure

Let us consider the measure K0 (P ||Q) given in (39) as forth measure. Here below we shall present some interesting inequalities having this measure. In view of (42) we have 1 1 1 0 6 K0 (P ||Q) − T (P ||Q) 6 Ψ(P ||Q) − T (P ||Q) 6 ΨM (P ||Q) − T (P ||Q), 8 16 16 i.e., 0 6 DK0 T (P ||Q) 6 DΨT (P ||Q) 6 DΨM T (P ||Q).

(49)

In view of (8) and (49) we have 1 1 0 6 K0 (P ||Q) − T (P ||Q) 6 K0 (P ||Q) − h(P ||Q), 8 8 i.e., 0 6 DK0 T (P ||Q) 6 DK0 h (P ||Q).

(50)

We shall now combine the inequalities (49) and (50)6. In order to do so, we need to establish an inequality between DK0 h (P ||Q) and DΨT (P ||Q). It is given in the proposition below. Proposition 2.4. The following inequalities hold: 1 DK0 h (P ||Q) 6 DΨT (P ||Q) 2

(51)


where

X 1 DK0 h (P ||Q) = K0 (P ||Q) − h(P ||Q = qi fK0 h 8 i=1 √ (x − 1)2 ( x − 1)2 √ fK0 h (x) = − 8 x 2

This gives us fK′ 0 h (x) =

3x2 − 8x3/2 + 6x − 1 , x>0 16x3/2 11

pi , qi

(52)

and fK′′ 0 h (x) =

3(x − 1)2 . 32x5/2

(53)

Thus we have fK′′ 0 h (x) > 0 for all x > 0, and hence, fK0 h (x) is convex for all x > 0. Also, we have fK0 h (1) = 0. In view of this we can say that the measure DK0 h (P ||Q) given by (52) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn ×Γn . The convexity of the measure DΨT (P ||Q) is already given in the Proposition 2.2. Let us consider now the following function √ fK′′ 0 h (x) 3(x + 1) x gK0h ΨT (x) = ′′ = , fΨT (x) 4(x2 + x + 1)

x > 0,

′′ where fK′′ 0 h (x) and fΨT (x) and are as given by (53) and (27) respectively.

Calculating the first order derivative of the function gK0 h ΨT (x) with respect to x, one gets 3(x − 1)(x2 + 3x + 1) > 0, x < 1 ′ √ gK0 h ΨT (x) = − . (54) 2 2 < 0, x > 1 8(x + x + 1) x In view of (54) we conclude that the function gK0 h ΨT (x) is increasing in x ∈ (0, 1) and decreasing in x ∈ (1, ∞), and hence 1 M = sup gK0 h ΨT (x) = gK0 h ΨT (1) = . 2 x∈(0,∞)

(55)

By the application of (14) with (55) we get (51).

2.5

Unification Inequalities

Proposition 2.5. The following inequalities hold: 1 0 6 DK0 T (P ||Q) 6 DK0 h (P ||Q) 6 DΨT (P ||Q) 6 2 6

1 1 ∗ D (P ||Q) 6 DΨM T (P ||Q). 128 4

(56)

Proof. In view of (49), (50) and (51), we have 1 1 0 6 DK0 T (P ||Q) 6 DK0 h (P ||Q) 6 DΨT (P ||Q) 6 DΨM T (P ||Q). 2 2

12

(57)

In view of Proposition 2.3, we have 1 ∗ 1 D (P ||Q). 0 6 DK0 T (P ||Q) 6 DK0 h (P ||Q) 6 DΨT (P ||Q) 6 2 128

(58)

We observe that the inequalities (57) and (58) are the same except in the last term in the right side. Below we shall give a relationship with DΨM T (P ||Q) and D ∗ (P ||Q). In view of (41), we can write 1 − K0 (P ||Q) 6 −T (P ||Q). 8 This gives 2ΨM (P ||Q) − 4K0 (P ||Q) 6 2 ΨM (P ||Q) − 32 T (P ||Q).

(59)

Now (37) and (59) together give ∗

D (P ||Q) 6 32 i.e,

1 ΨM (P ||Q) − T (P ||Q) , 16

1 ∗ D (P ||Q) 6 DΨM T (P ||Q). 32

(60)

Finally, (58), (59) and (60) together completes the proof.

3

Exponential Divergence

Let consider the following general measure Kt (P ||Q) =

n X (pi − qi )2(t+1) i=1

(pi qi )

2t+1 2

,

t = 0, 1, 2, 3, ...

(61)

When t = 0, we have the same measure as given in (39) and when t = 1, we have K1 (P ||Q) = D ∗ (P ||Q). When 2t + 1 = k, it reduces to one studied by Jain and Srivastava [4]. Now we shall prove the convexity of the measure (61).

13

We can write Kt (P ||Q) = where ft (x) =

n X i=1

pi qi ft , qi

(x − 1)2(t+1) x

2t+1 2

,

x>0

Calculating the first and second order derivative of ft (x), we get ft′ (x) =

(x − 1)2t+1 (2tx + 3x + 2t + 1) 2x

2t+3 2

and

ft′′ (x) =

(x − 1)2t (2t + 1) [(2t + 3)(x2 + 1) + 2(2t + 1)x] x

2t+5 2

,

∀x > 0,

t ∈ N.

(62)

Thus we have ft′′ (x) > 0 for all x > 0, and hence, ft (x) is convex for all x > 0 and t ∈ N. Also, we have ft (1) = 0. In view of this we can say that the measure Kt (P ||Q), t ∈ N given by (61) is nonnegative and convex in the pair of probability distributions (P, Q) ∈ Γn × Γn . For t = 0, 1, 2, 3, ... and x > 0 we have the following sequence of convex functions f0 (x) =

(x − 1)2 , x1/2

f1 (x) = fD∗ (x) =

(x − 1)4 , x3/2

(x − 1)6 , x5/2 (x − 1)8 , f3 (x) = x7/2 .. .

f2 (x) =

Let write a linear combination of convex functions, fEK (x) = c0 f0 (x) + c1 f1 (x) + c2 f2 (x)+ i.e., fEK (x) = c0

(x − 1)2 (x − 1)4 (x − 1)6 (x − 1)8 + c + c + c + ..., 1 2 3 x1/2 x3/2 x5/2 x7/2

14

where c0 , c1 , c2 , c3 , ...are the constants. For simplicity let us consider, c0 =

1 1 1 1 , c1 = , c2 = , c3 = , ... 0! 1! 2! 3!

Thus we have 1 (x − 1)2 1 (x − 1)4 1 (x − 1)6 1 (x − 1)8 + + + + ... 0! x1/2 1! x3/2 2! x5/2 3! x7/2 1 (x − 1)2 1 (x − 1)4 1 (x − 1)6 (x − 1)2 1 + + + + ... = x1/2 0! 1! x 2! x2 3! x3 # " 1 2 (x − 1)2 1 1 (x − 1)2 1 (x − 1)2 = + + ... . + x1/2 0! 1! x 2! x

fEK (x) =

(63)

This gives us (x − 1)2 exp fEK (x) = x1/2

(x − 1)2 x

.

As a consequence of (64), we have the following divergence measure n X pi EK (P ||Q) = qi fEK qi i=1 n X (pi − qi )2 (pi − qi )2 = exp , (P, Q) ∈ Γn × Γn , √ p q p q i i i i i=1

(64)

(65)

The measure (65) has been presented by Jain and Srivastava [4]. We shall call it exponential divergence. As a consequence of the expression (63), it is obvious that K0 (P ||Q) + D ∗ (P ||Q) 6 EK (P ||Q),

(66)

where D ∗ (P ||Q) = K1 (P ||Q). From the inequalities (8) and (42), we have 1 1 ∆(P ||Q) 6 I(P ||Q) 6 h(P ||Q) 6 J(P ||Q) 6 T (P ||Q) 6 4 8 1 1 1 6 K0 (P ||Q) 6 Ψ(P ||Q) 6 ΨM (P ||Q). 8 16 16

(67)

Here below we shall relate the measure ΨM (P ||Q) with exponential divergence EK (P ||Q). 15

In view of (37), we have 1 1 ΨM (P ||Q) = K0 (P ||Q) + D ∗ (P ||Q) 6 K0 (P ||Q) + D ∗ (P ||Q). 2 4

(68)

Now (68) together with (66) gives 1 ΨM (P ||Q) 6 EK (P ||Q). 2 Inequalities (67) and (69) together give 1 1 ∆(P ||Q) 6 I(P ||Q) 6 h(P ||Q) 6 J(P ||Q) 6 T (P ||Q) 6 4 8 1 1 1 1 6 K0 (P ||Q) 6 Ψ(P ||Q) 6 ΨM (P ||Q) 6 EK (P ||Q). 8 16 16 8

(69)

(70)

Also we have 1 1 ∗ 1 DK0 T (P ||Q) 6 DK0 h (P ||Q) 6 DΨT (P ||Q) 6 D (P ||Q) 6 EK (P ||Q). 2 128 128

(71)

Moreover in view of (48) and (69) the following inequalities among the new measures hold: FN (P ||Q) 6

4

1 1 1 ∗ D (P ||Q) 6 ΨM (P ||Q) 6 EK (P ||Q). 192 96 48

(72)

Bounds on New Divergence Measures

In this section we shall give bounds on the measures studied in sections 2 and 3. These bounds are based on the theorem given below. Theorem 4.1. Let f : R+ → R be differentiable convex and normalized i.e., f (1) = 0. If P, Q ∈ Γn , 0 ≤ Cf (P ||Q) ≤ WCf (P ||Q),

(73)

where WCf (P ||Q) =

n X i=1

16

pi (pi − qi )f ′ ( ), qi

(74)

In addition, if we have 0 < r ≤ pqii ≤ R < ∞, ∀i ∈ {1, 2, ..., n}, for some r and R with 0 < r ≤ 1 ≤ R < ∞, then the followings hold: and

0 ≤ Cf (P ||Q) ≤ WCf (P ||Q) ≤ ZCf (r, R),

(75)

0 ≤ Cf (P ||Q) ≤ YCf (r, R) ≤ ZCf (r, R),

(76)

1 ZCf (r, R) = (R − r) [f ′ (R) − f ′ (r)] , 4

(77)

where

and YCf (r, R) =

(R − 1)f (r) + (1 − r)f (R) . R−r

(78)

More details on the above theorem refer to Taneja [10]. Here below we shall give examples of the inequality (73). This we have done for the measures EK (P ||Q), Kt (P ||Q), FN (P ||Q), and ΨM (P ||Q). The measures D ∗ (P ||Q) and K0 (P ||Q) are the particular cases of Kt (P ||Q). The applications of the inequalities (74) and (75) can be obtained on similar lines. Example 4.1. Let us consider the measure Kt (P ||Q) given by (61). Then applying the inequality (73), we have the following bound: 0 ≤ Kt (P ||Q) ≤ WKt (P ||Q),

where

WKt (P ||Q) =

2(t+1) n X qi pi − qi qi

i=1

2t+3 2

pi

t = 0, 1, 2, ...

(79)

(2t + 3)pi + (2t + 1)qi , t = 0, 1, 2, ... 2

In particular we have

and

2 3/2 n X qi 3pi + qi pi − qi , 0 ≤ K0 (P ||Q) ≤ q p 2 i i i=1

(80)

4 5/2 n X qi 5pi + 3qi pi − qi

(81)

∗

0 ≤ D (P ||Q) ≤

i=1

qi

17

pi

2

.

Inequalities (80) and (81) are obtained by considering t = 0 and t = 1in (78) respectively. The measures K0 (P ||Q) and D ∗ (P ||Q) are the same as given by (39) and (9) respectively. Example 4.2. Let us consider the measure EK (P ||Q) given by (65). Then applying the inequality (73), we have the following bound: 0 ≤ EK (P ||Q) ≤ WEK (P ||Q),

(82)

where

WEK (P ||Q) =

n X

(pi − qi )2 [qi (pi − qi )2 + 2p3i + pi qi2 + qi3 ] 5/2 3/2

2pi qi

i=1

!

exp

(pi − qi )2 . pi qi

Example 4.3. Let us consider the measure ΨM (P ||Q) given by (38). Then applying the inequality (73), we have the following bound: 0 ≤ ΨM (P ||Q) ≤ EΨM (P ||Q),

(83)

where EΨM (P ||Q) =

n X (pi − qi )4 (pi + qi )(5p2 + 3q 2 ) i

5/2 3/2 2pi qi

i=1

i

.

Example 4.4. Let us consider the measure FN (P ||Q) given by (44). Then applying the inequality (73), we have the following bound: 0 ≤ FN (P ||Q) ≤ WFN (P ||Q),

(84)

where √ √ 4 √ √ √ n X pi − qi pi + qi pi + 2qi + pi qi EFN (P ||Q) = . √ (pi + qi )2 pi i=1 As we mentioned above we can also obtain bounds applying the inequalities (74) and (75) by simple calculations. The results obtained shall be in terms of r and R.

18

References [1] J. BURBEA, and C.R. RAO, On the convexity of some divergence measures based on entropy functions, IEEE Trans. on Inform. Theory, IT-28(1982), 489-495. ´ Information type measures of differences of probability distribution and [2] I. CSISZAR, indirect observations, Studia Math. Hungarica, 2(1967), 299-318. [3] S. S. DRAGOMIR, J. SUNDE and C. BUS¸E, New inequalities for jeffreys divergence measure, Tamsui Oxford Journal of Mathematical Sciences, 16(2)(2000), 295-309. [4] K.C. JAIN AND A. SRIVASTAVA, On Symmetric Information Divergence Measures of Csiszars f - Divergence Class, Journal of Applied Mathematics, Statistics and Informatics (JAMSI), 3(1)(2007), 85-102. [5] H. JEFFREYS, An invariant form for the prior probability in estimation problems, Proc. Roy. Soc. Lon., Ser. A, 186(1946), 453-461. [6] S. KULLBACK and R.A. LEIBLER, On information and sufficiency, Ann. Math. Statist., 22(1951), 79-86. [7] P. KUMAR and A. JOHNSON, On A Symmetric Divergence Measure and Information Inequalities, Journal of Inequalities in Pure and Appl. Math., 6(3)(2005), 1-13. [8] I.J. TANEJA, New developments in generalized information measures, Chapter in: Advances in Imaging and Electron Physics, Ed. P.W. Hawkes, 91(1995), 37-135. [9] I.J. TANEJA, Generalized information measures and their applications, on line book: http://www.mtm.ufsc.br/∼taneja/book/book.html, 2001. [10] I.J. TANEJA, On symmetric and non-symmetric divergence measures and their generalizations, Chapter in: Advances in Imaging and Electron Physics, Ed. P.W. Hawkes, 138(2005), 177-250. [11] I.J. TANEJA, Refinement Inequalities Among Symmetric Divergence Measures, The Australian Journal of Mathematical Analysis and Applications, 2(1)(2005), Art. 8, pp. 1-23. [12] I.J. TANEJA and P. KUMAR, Relative Information of Type s, Csiszar’s f-Divergence, and Information Inequalities, Information Sciences, 166(2004), 105-125.

19