Features of constrained entropic functional variational ...

Features of constrained entropic functional variational problems A. R. Plastino1 , A. Plastino2,3 , M.C.Rocca2,3 2

Departamento de F´ısica, Fac. de Ciencias Exactas, Universidad Nacional de La Plata C.C. 727 (1900) La Plata, Argentina 3 IFLP-CCT-CONICET-C.C 727 (1900) La Plata. Argentina 1 CeBio y Secretaria de Investigacion, Univ. Nac. del Noroeste de la Prov. de Bs. As., UNNOBA and CONICET, R. Saenz Pe˜ na 456, Junin, Argentina (Dated: April 19, 2018) We describe in great generality features concerning constrained entropic, functional variational problems that allow for a broad range of applications. Our discussion encompasses not only entropies but, potentially, any functional of the probability distribution, like Fisher-information or relative entropies, etc. In particular, in dealing with generalized statistics in straightforward fashion one may sometimes find that the celebrated relation between entropic small changes and mean energy > = β d /dβ),

(1)

where β is the inverse temperature, S the entropy and U the internal energy. This relation, central to thermal physics, is closely related to thermodynamics’ first law. Deriving it is trivial in the case of Boltzmann-Gibbs’ logarithmic entropy. Just look it up in any text-book [21, 22]. However, for general entropies such is not the case (see, as one of many possibilities, Ref. [23]). Let us look in detail at a famous example so as to clearly illustrate the problem we are talking about. A.

A typical abeyance example

An example is appropriate to appreciate the difficulties we are here referring to. Our probability density functions (PDFs) are designed with the letter p, and pM E would stand for the MaxEnt PDF. We will use the q-functions [19] eq (x) = [1 + (1 − q)x]1/(1−q) ; eq (x) = exp (x) f or q = 1

(2)

2 ;

lnq (x) =

x(1−q) − 1 ; lnq (x) = ln (x) f or q = 1. 1−q

(3)

We define the Tsallis q- entropy, for any real q, as Z ST =

dxf (p),

(4)

f (p) =

p − pq . q−1

(5)

with

Our a priori knowledge is that of the mean energy < U > (canonical ensemble). The MaxEnt variational problem becomes, with Lagrange multipliers λN , λU 1 − qpq−1 − λU U − λN = 0, q−1

f 0 (p) =

1 − qpq−1 . q−1

(6)

(7)

One conveniently defines here g(p) as the inverse of f 0 (p) such that g[f 0 (p)] = p [25]. One has g(ν) = q 1−q [1 − (q − 1)p0 ]1/(q−1) = q 1−q e(2−q) (p0 ).

(8)

pM E = g(λN + λU U ),

(9)

pM E = g(λN + λU U ) = q 1−q e(2−q) (λN + λU U ),

(10)

It is obvious that

or

so that onecannot extract λN from that expression. Moreover, you do not obtain explicitly the relation between Z and λN . Since one can not immediately derive from it a value for λN , a heuristic alternative is to introduce λN = −

q 1 1 Z q−1 + = [1 − qZTq−1 ], q−1 T q−1 q−1

(11)

with ZT unknown for the time being, and re-express λU in the guise λU = qZT1−q β.

(12)

where β is determined by the above equation. The variational problem becomes 1 − qpq−1 q 1 =− ZT1−q + + ZT1−q qβU = 0 (q − 1) q−1 q−1

(13)

pq−1 = ZT1−q [1 − (q − 1)βU ],

(14)

and yields pM E = ZT−1 [1 − (q − 1)βU ]1/(q−1) ,

(15)

3 where β is definitely NOT the variational multiplier λU . Moreover, we can now have an expression for Z ZT = dx [1 − (q − 1)βU ]1/(q−1) .

(16)

Thus, we have pq p1−q = p; pq lnq (p) =

p − pq , 1−q

(17)

and then Z Z Sq = − dxpq lnq (p) = dxp[1 − pq−1 ]/(q − 1) = Z   = dxp 1 − (1/ZT )q−1 [1 − (q − 1)βU ] /(q − 1) = Z   = dxp [1 − (1/ZT )q−1 ]/(q − 1)] + (1/ZT )q−1 βU = Z h i = dxp lnq (ZT ) + ZT1−q βU . Sq = lnq (ZT ) + ZT1−q βhU i = lnq (ZT ) + βhU i/q,

(18) (19) (20) (21) (22)

entailing (2−q)/(1−q) 2−q eq (βU )

Z dSq = T dβ

1−q

+ hU i/q + β

∂hU i/q . ∂β

(23)

We encounter now, as a result, that Eq. (1) is violated for β. This is a fact that has created some confusion in the Literature [23]. A treatment similar to that above can be found in [24] for Renyi’s entropy.

B.

Our present goal

We will proceed, starting with the next Section, to overcome the difficulties posed by the above kind of situations. The paper is organized as follows. Section II contains a very general proof. It applies to any functional of the probability distribution, like generalized entropies, Fisher information, relative entropies, etc. We will demonstrate the fact that Eq. (1) always holds, no matter what the quantifier one has in mind might be, becoming in fact a basic result of the variational problem. In order to further clarify the issue at hand we specify this proof in several particular instances of interest. In Section III we revisit Tsallis’ quantifier. Renenyi’s entropy is discussed in Sect. IV. An arbitrary, trace form entropic quantifier is the focus of Section V and, finally, an also arbitrary entropic functional lacking trace form is examined in Sect. VI. Some concluions are drawn in Sect. VII.

II.

THE GENERAL VARIATIONAL PROBLEM A.

Functional derivatives: a brief reminder

A functional F of a distribution g is a mapping between a collection of g’s and a set of numbers [26]. The functional derivative can be introduced via the Taylor expansion Z f [g + h] = F [g] + 

dx

δF h(x) + O(2 ), δg(x)

(24)

δF for any reasonable h(x). Here δg(x) becomes the definition of a functional derivative. Note that it is both a function of x and a functional of g. In our case, generalized entropies constitute our foremost example of a functional.

4 B.

The general problem

Let F and G be functionals of a normalized probability density function (PDF) f . Z F = F [f ]; G = G[f ]; dxf (x) = 1.

(25)

Given two functionals F and G, one wishes to extremize F subject to a fixed value for G. The ensuing variational problem reads Z δ[F − bG − a

dx f ] ⇒

δG δF −b − a = 0, δf δf

(26)

∂f ∂f ∂a + ] = 0. ∂b ∂a ∂b

(27)

while Z

Z dxf (a, b, x) = 1 ⇒

dx[

Eq. (27) plays a very important role in our endeavors, as we will presently see. We now face Z

δF ∂f ∂f ∂a [ + ], δf ∂b ∂a ∂b

(28)

dF δG ∂f ∂f ∂a = [b + a][ + ]. db δf ∂b ∂a ∂b

(29)

dF = db

dx

so that, using (27), as just promised

Use now f −normalization to derive the fundamental relation δG ∂f ∂f ∂a dF = [b ][ + ] = b(dG/db). db δf ∂b ∂a ∂b

(30)

QED. The theme has been broached in different manners to ours, for example, in [25, 38, 39], but without (i) our specific details and (2) our generality. For further clarification we address below important particular cases. III.

TSALLIS’ MAXENT VARIATIONAL PROBLEM REVISITED

Since the Lagrange muktipliers are the focus of the problems we are trying to solve, we change notations and call them simply a, b. We have for ST Z δ

 dx

f − fq + bU f + af q−1

 = 0,

(31)

and then qf q−1 = 1 − (q − 1)(a + bU ),

(32)

so that Tsallis’ canonical MaxEnt distribution f with linear constraints is

f =[

1 − [(q − 1)(a + bU (x))] 1/(q−1) ] , q

(33)

with a, b Lagrange multipliers, b the inverse temperature T . The first Law states that dS d =b . db db

(34)

5 Now set G = (1 − [(q − 1)(a + bU )])

(2−q)/(q−1)

,

1 Q(q) = ( )1/(q−1) , q

(36)

1 , q−1

(37)

da + U ], db

(38)

D=

K=[

(35)

entailing (df /db) = QDGK,

(39)

and, because of f −normalization, we derive the fundamental relation Z QD

dxGK = 0.

(40)

Tsallis entropy is R 1 − dxf q , S= q−1

(41)

so that dS = −Dq db

Z

dxf q−1 DGK,

(42)

but, since f q−1 = (1/q) (1 − [(q − 1)(a + bU )]) .

(43)

Accordingly, dS = −QD2 db

Z dx (1 − [(q − 1)(a + bU )]) GK,

(44)

that we decompose so as to take advantage of Eq. (40). dS = QD2 db

Z dx [GK[(q − 1)(a + bU )]] ,

(45)

and re-using Eq. (40) dS = QD db Now:

Z dxbU GK.

(46)

6

d = db

Z

df , db

(47)

dxU GK.

(48)

dxU

that is d = QD db

Z

Comparing Eq. (48) with Eq. (46) we see that dS d =b . db db

(49)

QED. IV.

THE CASE OF RENYI’S ENTROPY

Renyi’s quantifier SR is an important quantity in several areas of scientific effort. One can cite as examples ecology, quantum information, the Heisenberg XY spin chain model, theoretical computer science, conformal field theory, quantum quenching, diffusion processes, etc. [27–36], and references therein. An important Renyi-characteristic lies in its lack of trace form. We have 1 SR = ln 1−q

Z

 f dx = q

Z

1 ln J, 1−q

f q dx,

J=

(50)

(51)

with dJ =q db

Z

f q−1

df dx. db

(52)

The variational problem is  Z δ ln dx

fq (1 − q)



Z −

 dx [bU f + af ] = 0,

(53)

yielding qf q−1 − bU − a = 0, J(1 − q)

(54)

i.e., f q−1 =

J(1 − q) [a + bU ], q

(55)

and  f=

J(1 − q) [a + bU ] q

1/(q−1) ,

(56)

7 is the MaxEnt solution, with a, b Lagrange multipliers, b the inverse temperature T . df 1 = db q−1



(2−q)/(q−1)   J(q − 1) J(1 − q) da dJ [a + bU ] [ + U ] + (1/q)[a + bU ](q − 1) , q q db db

 G=

 K=

J(1 − q) [a + bU ] q

(57)

(2−q)/(q−1) ,

dJ J(1 − q) da [ + U ] + (1/q)[a + bU ](q − 1) q db db

(58)

 .

(59)

Now: df 1 = GK, db q−1

(60)

so that, on account of f −normalization, we derive the fundamental relation Z dxGK = 0.

(61)

The first Law states that d dSR =b , db db

(62)

with Z

d = db

df , db

(63)

dx U G K.

(64)

df , db

(65)

dx U

that is d 1 = db q−1

Z

According to (50) dSr q = db J(1 − q)

Z

dx f q−1

i.e., dSr q = db J(1 − q)

Z dx

df J(1 − q) [a + bU ], db q

(66)

or, dSr 1 = db q−1 and using (61)

Z dx [a + bU ]G K,

(67)

8

dSr 1 = db q−1

Z dx bU G K.

(68)

Comparing with (64) we obtain dSR d =b , db db

(69)

QED. V.

GENERAL ENTROPIES OF TRACE FORM

 dxR[f (x)] ,

Z S=

(70)

with R an arbitrary smooth function. Then Z

0

S =

dxR0 .

(71)

(Here R0 denotes the functional derivative). The MaxEnt variational problem is Z δ

Z

 dx (R − af − bU f ) = 0.

(72)

 dx (R0 − a − bU ) = 0.

(73)

R0 = a + bU

(74)

g = (R0 )(−1) ; so that g[R0 ] = f = R0 [g].

(75)

f = g[a + bU ]

(76)

dyg[a + bU ] = 1.

(77)

Define now the inverse function if R0

Z

d db

Z

Z

dx g 0 [

d = db

Z

dxg[a + bU ] = 0.

(78)

da + U ] = 0. db

(79)

dx g 0 [a + bU ][

da + U ]U. db

(80)

9 We use now (75) to set R0 [g] = a + bU , and then dS = db

Z

dR dx = db

Z

dg dx R [g] = db 0

Z

dx (a + bU )g 0 [

da + U ]. db

(81)

We use now normalization (79) and obtain the fundamental relation dS = db

Z

dx (a + bU )g 0 [a + bU ][

da + U] = db

Z

dx bU g 0 [a + bU ][

da + U ], db

(82)

so that comparing (80) with (82) we satisfy the first Law. VI.

GENERAL ENTROPIES LACKING TRACE FORM

Z S=B

 dxR[f ]

,

(83)

R with B an arbitrary smooth functional. Define the number J = B 0 [ dxR(f )]. dS =J db

Z

dxR0 (df /db).

(84)

(Here S 0 denotes the functional derivative). Define F = R0 and consider the inverse function of F , namely, g = F (−1) ; F [g(f )] = f ; g[F (f )] = f.

(85)

The MaxEnt variational problem ends up being JF (f ) − a − bU = 0,

(86)

f = g[(a + bU )/J],

(87)

so that the MaxEnt solution’s PD fM E is

and the MaxEnt entropy reads Z SM E = B

 dxR[g({a + bU )}/J)] .

(88)

One also has

0=

Z

d db



Z dxf.

∂a dx g [(a + bU )/J] [ + U ]J −1 − (1/J 2 )(dJ/db)[a + bU ] ∂b 0

(89)

 = 0,

(90)

Now, d = db

Z

 ∂a −1 2 dx U g ([a + bU ]/J) [ + U ]J − (1/J )(dJ/db)[a + bU ] . ∂b 0



(91)

10 0

We will use now (85) to set F [g] = (a + bU )/J.   ∂a dx (a + bU )J −1 g 0 [ + U ]J −1 − (1/J 2 )(dJ/db)[a + bu] . ∂b (92) We use now f −normalization and derive the fundamental relation dS =J db

Z

0

Z

0

dx R [{g(a + bU )/J}]g [(a + bU )/J] = J

dS =J db

Z



 ∂a −1 2 dx [(a + bU )/J]g [(a + bU )/J] [ + U ]J − (1/J )(dJ/db)[a + bU ] , ∂b 0

(93)

so that Z

 ∂a −1 2 + U ]J − (1/J )(dJ/db)[a + bU ] , dx bU g [(a + bU )/J] [ ∂b 0



(94)

so that comparing (91) with (94) we satisfy the first Law.

VII.

CONCLUSIONS

d We have conclusively shown that the thermal relation dS is obeyed by any system subject to a Legendre dβ = β dβ extremization process, i.e., in any constrained entropic variational problems, no matter what form the entropy adopts and what kind of constraints are used, We will demonstrate the fact that Eq. (1) always holds, no matter what the quantifier one has in mind might be. The essential tool of our proofs is a judicious use of the normalization requirement.

Note that the treatment of Section IIB encompasses the three different forms of non-linear averaging that have been proposed for Tsallis’ statistics in [40].

11

[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19]

[20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37]

[38] [39] [40]

D. Bagchi and C. Tsallis, Physica A 491 (2018) 869. L.J.L Cirto, V.R.V. Assis and C. Tsallis, Physica A 393 (2014) 286. A.M.C. Souza, R.F.S. Andrade, F.D. Nobre, and E.M.F. Curado, Physica A 491 (2018) 153. B.S. Chahal, M. Singh, Shalini and N.S. Saini, Physica A 491 (2018) 935. A.S. Bains, M. Tribeche, N.S. Saini and T.S. Gill, Physica A 466 (2017) 111. E.P. Bento, J.R.P. Silva and R. Silva, Physica A 392 (2013) 666. J.C. Carvalho, B.B. Soares, B.L. Canto Martins, J.D. Do Nascimento, A. Recio-Blanco and J.R. de Medeiros, Physica A 384 (2007) 507. A. Carati, Physica A 387 (2008) 1491. G. Ruiz and C. Tsallis, Physica A 386 (2007) 720. U. Tirnakli and E.P. Borges, Nature Scientific Reports 6 (2016) 23644. A. Guha and P.K. Das, Physica A 497 (2018) 272. A.M.C. Souza, Physica A 390 (2011) 2686. L.G.A. Alves, H.V. Ribeiro, M.A.F. Santos, R.S. Mendes, and E.K. Lenzi, Physica A 429 (2015) 35. H.C. Soares, J.B. Meireles, A.O. Castro, J.A.O. Huguenin, A.G.M. Schmidt, and L. da Silva, Physica A 432 (2015) 1. P. Quarati andf A.M. Scarfone, Physica A 392 (2013) 6512. T.D. Frank, Physica A 388 (2009) 2503. E.K. Lenzi and A.M. Scarfone, Physica A 391 (2012) 2543. A.M. Scarfone, H. Matsuzoe and T. Wada, Phys. Lett. A 380 (2016) 3022. M. Gell-Mann and C. Tsallis, Eds. Nonextensive Entropy: Interdisciplinary applications, Oxford University Press, Oxford, 2004; C. Tsallis, Introduction to Nonextensive Statistical Mechanics: Approaching a Complex World, Springer, New York, 2009. See http://tsallis.cat.cbpf.br/biblio.htm for a regularly updated bibliography on the subject. R. B. Lindsay, H. Margenau, Foundations of physics (Dover, NY, 1957); F. Reif, Fundamentals of Statistical and Thermal Physics (McGraw-Hill, New York NY, 1965). R. K. Pathria, Statistical Mechanics (Pergamon, Exeter, 1993). S. Abe, S. Martinez, F. Pennini, A. Plastino, Phys. Lett. A 281 (2001) 126. A. Plastino, M.C. Rocca, G.L. Ferri, Physica A 505 (2018) 794. A. Plastino, A. R. Plastino, Phys. Lett. A 226 (1997) 257. G. Parisi, Statistical field theory (Addison Wesley, New york, 1988). C. M. Herdman, Stephen Inglis, P.-N. Roy, R. G. Melko, and A. Del Maestro, Phys. Rev. E 90, 013308 (2014). Mohammad H. Ansari and Yuli V. Nazarov, Phys. Rev. B 91, 174307 (2015). Lei Wang and Matthias Troyer, Phys. Rev. Lett. 113, 110401 (2014). Matthew B. Hastings, Ivn Gonzlez, Ann B. Kallin, and Roger G. Melko, Phys. Rev. Lett 104, 157201 (2010). Richard Berkovits, Phys. Rev. Lett. 115, 206401 (2015). Nima Lashkari, Phys. Rev. Lett. 113, 051602 (2014). Gabor B. Halasz and Alioscia Hamma, Phys. Rev. Lett. 110, 170605 (2013). MB Hastings, I Gonzlez, AB Kallin, RG Melko, Phys. Rev. Lett. 104, 157201 (2010); A. De Gregorio, S.M. lacus, 179, 279 (2009). Leila Golshani, Einollah Pasha, Gholamhossein Yari, Information Sciences, 179, 2426 (2009); J.F. Bercher, Information Sciences 178, 2489 (2008). EK Lenzi, RS Mendes, LR da Silva, Physica A 280 (2000) 337. J. Skilling, Massive Inference and Maximum Entropy, in Maximum Entropy and Bayesian Methods, pages 1-14, 1997 Proceedings of the 17th International Workshop on Maximum Entropy and Bayesian Methods of Statistical Analysis, Editors J. Erickson, Joshua T. Rychert, C. Ray Smith (Springer, New York). E. M. F. Curado, Braz. J. Phys. 29 (1999) 36. R.S. Mendes, Physica A 242 (1997) 299. E. M. F. Curado, C. Tsallis C, J. Phys. A: Math. Gen. 24 (1991) L69; C. Tsallis, R. S. Mendes, A. R. Plastino, Physica A 261 (1998) 534; S. Martnez, F. Nicolas, F. Pennini, A. Plastino, Physica A 286 (2000) 489.