On an open problem for dual symmetric divergences Shigeru Furuichi1 , Flavia-Corina Mitroi-Symeonidis2y 1
Department of Computer Science and System Analysis, College of Humanities and Sciences, Nihon University, 3-25-40, Sakurajyousui, Setagaya-ku, Tokyo, 156-8550, Japan 2 Department of Mathematical Methods and Models, Faculty of Applied Sciences, University Politehnica of Bucharest, Romania
Abstract. We discuss the necessity of a restriction suggested by other authors in their attempt to solve the problem. Keywords : Mathematical inequality, Tsallis relative entropy, Je¤reys divergence, JensenShannon divergence 2010 Mathematics Subject Classi…cation : 26D15 and 94A17
1
Introduction
First, let us recall that for the study of multifractals, in 1988, Tsallis [10] introduced a oneparameter generalization of Shannon entropy by n X
Hq (p)
pqj lnq pj =
j=1
n X
pj lnq
j=1
1 ; (q pj
0; q 6= 1)
(1)
where p = fp1 ; p2 ; ; pn g is a probability distribution with pj > 0 for all j = 1; 2; ; n and x1 q 1 the q logarithmic function for x > 0 is de…ned by lnq (x) which uniformly converges 1 q to the usual logarithmic function log(x) in the limit q ! 1. Therefore Tsallis entropy converges to Shannon entropy in the limit q ! 1: n X
lim Hq (p) = H1 (p)
q!1
For two probability distributions p = fp1 ; p2 ; Dq (pjjr)
pj log pj :
(2)
j=1
n X
pqj (lnq
pj
; pn g and r = fr1 ; r2 ; lnq rj ) =
j=1
n X j=1
pj lnq
rj pj
; rn g we denote by (3)
the Tsallis relative entropy, which converges to the usual relative entropy (divergence, KullbackLeibler information) in the limit q ! 1: lim Dq (pjjr) = D1 (pjjr)
q!1
y
n X j=1
E-mail:
[email protected] E-mail:
[email protected]
1
pj (log pj
log rj ):
(4)
For q 0 with q 6= 1, the q-exponential function is the inverse function of the q-logarithmic function, de…ned by expq (x) f1 + (1 q)xg1=(1 q) ; if 1 + (1
q)x > 0, otherwise it is unde…ned.
Lemma 1.1 ([5], Lemma 3.3) The function f (x) = is concave for 0
r
lnr
1 + expq ( x) 2
q.
Remark 1.2 A closer look at our result reveals that there is no need to add in this statement the condition "q > 1" as suggested in a recent work [8, Lemma 1], since we take into account that the function expq ( x) already comes with its more general existence condition 1 + (1
q) ( x) > 0:
(5)
This condition enabled us to establish the above Lemma for f on an adequate domain. As about the case considered in [8], x = 3; r = 1=5; q = 1=3, it does not satisfy the condition (5), so it provides no counterexample. We emphasize that by putting the condition q > 1 many other valid cases are not considered, 1+expq ( x) for instance all the functions f (x) = lnr are concave on 1; 1 1 q for all …xed 2 values 0 r q < 1. See the pictures below. We considered the maximal domain of de…nition for each case (in other words, its upper bound depends on the values of q): The origin (0; 0) is a common point for all nonnegative r; q: 1. When q changes, the domain of de…nition changes too (r is constant). 1 0.5
f(x)
0 -0.5 -1 r = 0.2 , q = 0.2 r = 0.2 , q = 0.4 r = 0.2 , q = 0.7
-1.5 -2 -2
-1
0
1
2
3
4
-2 < x < 1/(1-q)
2. When r changes, the domain of de…nition remains the same, picture shows three cases (we kept q constant).
2
1; 1 1 q . The following
1 0.5
f(x)
0 -0.5 -1 -1.5 q = 0.8 , r = 0.8 q = 0.8 , r = 0.6 q = 0.8 , r = 0.1
-2 -2.5 -2
-1
0
1
2
3
4
5
-2 < x < 1/(1-q)
Remark 1.3 Clearly the trivial limit case r = q; q ! 1 leads to the function f (x) =
log
1 + exp ( x) 2
which is also concave on R:
2
Main results
We review the de…nitions of two famous divergences. De…nition 2.1 ([2],[6]) The Je¤ reys divergence is de…ned by J1 (pjjr)
D1 (pjjr) + D1 (rjjp)
(6)
and the Jensen-Shannon divergence is de…ned as 1 p+r + D1 rjj 2 2
1 p+r D1 pjj 2 2
JS1 (pjjr)
:
(7)
Analogously we de…ne the following divergences. De…nition 2.2 ([5]) The Je¤ reys-Tsallis divergence is Jr (pjjr)
Dr (pjjr) + Dr (rjjp)
(8)
and the Jensen-Shannon-Tsallis divergence is JSr (pjjr)
1 p+r Dr pjj 2 2
1 p+r + Dr rjj 2 2
:
(9)
For the same reasons as above we reject the suggestion to add the condition q > 1 (see [8], [7]) in the statement of the following theorem. Theorem 2.3 ([5]) It holds that JSr (pjjr) for 0
r
min
(
lnr
1 2 Jq (pjjr)
1 + expq 2
q. 3
) 1 ; J 1+r (pjjr) 4 2
(10)
Remark 2.4 ([5]) For q = r; r ! 1 we captured the main result in [1]: JS1 (pjjr)
log
1 2 J1 (pjjr)
1 + exp
1 J1 (pjjr): 4
2
(11)
De…nition 2.5 The dual symmetric Je¤ reys-Tsallis divergence and the dual symmetric JensenShannon-Tsallis divergence are de…ned by Jr(ds) (pjjr)
Dr (pjjr) + D2
r (rjjp)
(12)
respectively JSr(ds) (pjjr)
p+r 1 Dr pjj 2 2
+ D2
r
rjj
p+r 2 (ds)
As one can see directly from the de…nition, we …nd that Jr (ds) JSr (pjjr)
:
(13) (ds) r (rjjp)
(pjjr) = J2
(ds) JS2 r (rjjp).
= See [9] and references therein for additive duality r $ 2 Tsallis statistics. (ds) Then we have the following upper bound for JSr (pjjr).
and r in
Theorem 2.6 ([5]) The following inequality holds
for all 1 < r
n o max JSr(ds) (rjjp); JSr(ds) (pjjr)
2 and r
lnr
1 2 Jq (pjjr)
1 + expq 2
;
(14)
q:
(ds)
(ds)
(ds)
Remark 2.7 For q = r; r ! 1 we have JSr (pjjr) ! JS1 (pjjr) = JS1 (pjjr); Jr (pjjr) ! (ds) J1 (pjjr) = J1 (pjjr) and the inequality (14) yields again the left side inequality of (11). In what follows we discuss about an open problem which we proposed during our talk at The Seventh Congress of Romanian Mathematicians, Bra¸sov (June 29-July 5, 2011), open problem published in [5]. Open problem 2.8 ([5]) Prove, disprove or …nd conditions such that the following inequality holds: 1 (ds) 1 + expr (pjjr) 2 Jr (ds) JSr (pjjr) lnr ; r 2 [0; 2]nf1g: (15) 2 In [8, Theorem 5 ] there is an attempt to positively solve it. For readers’ convenience we quote that result it here: Theorem 2.9 (Theorem 5, [8]) JSr(ds) (pjjr)
lnr
JSr(ds) (pjjr)
lnr
1 + expq1 1 + expq2
1 2
(Dq1 (pjjr) + Dq2 (rjjp)) ; 0 2 1 2 (Dq1 (pjjr) + Dq2 (rjjp)) ; 0 2
2
r
