Shannon Entropy for the Feller-Pareto (FP) Family and Order Statistics of FP Subfamilies

Transcript

1 Applied Mathematical Sciences, Vol. 4, 200, no. 0, Shannon Entropy for the Feller-Pareto (FP) Family and Order Statistics of FP Subfamilies S. Tahmasebi and J. Behboodian 2 Department of Statistics Shiraz University, Shiraz 7454, Iran stahmasby@yahoo.com 2 Department of Mathematics Shiraz Islamic Azad University, Shiraz, Iran Behboodian@stat.susc.ac.ir Abstract. In this paper, we derive the exact analytical expressions of entropy for the Feller- Pareto(FP) family and order statistics of FP subfamilies. The FP family is a very general unimodal distribution which includes a hierarchy of Pareto models, Transformed beta family, Burr families, Generalized Pareto, and Inverse Burr distributions. These distributions apply in reliability, actuarial science, economics, finance and telecommunications.we also present entropy ordering property for the sample minimum and maximum of FP subfamilies. Keywords: Burr distribution, Entropy, Feller-Pareto distribution, Order statistics, Pareto models, Transformed beta family. Introduction: The concept of entropy originated in the nineteenth century by Shannon []. During the last sixty years, a number of research papers and monographs discussed and extended Shannon s original work. Renyi [0], Cover and Tomas [3], Lazo and Rathie [8], Kapur [5], Kullback [7], are among the researchers in this area.also,the Shanon information of order statistics have been studied by a few authors.among them Wong and Chen [2], Park [9], provided some results of Shannon entropy for order statistics.the Shannon entropy for a continuous random variable X with probability density function f X (x) is defined as H(X) = f X (x) log f X (x)dx = 0 log f X (FX (u))du. () This is a mathematical measure of information which measures the average reduction of uncertainty of X.The aim of this paper is to determine the exact form of the Shannon information for the Feller-Pareto family and order statistics of FP subfamilies.the FP family includes a variety of unimodal distributions,it includes a hierarchy of Pareto models, Transformed beta

2 496 S. Tahmasebi and J. Behboodian family, Generalized Pareto, and Inverse Burr distributions.these distributions arise as tractable parametric models in reliability, actuarial science, economics, finance, telecommunications.the rest of this paper is organized as follows. In Section 2, we describe two different representations of the FP family and some general distributions which the FP family includes as the special cases.also, we provide a technique by one-dimensional integral to calculate the entropy measure of the FP, Pareto (IV), Inverse Burr, and Generalized Pareto Distributions.In Section 3,we present Shannon entropy for jth order statistic and entropy ordering property for the sample minimum and maximum of FP subfamilies. 2. Entropy for the Feller-Pareto and related distributions: The FP family traces its roots back to Feller [4], but the form we consider here was first defined and investigated by Arnold and Laguna [2]. Feller [4] defined the FP family as follows. Definition 2.: The FP distribution is defined as the distribution of X = μ + (Y ), where Y Beta(λ,λ 2 ),>0, and >0, So we write X FP(μ,,, λ,λ 2 ). The pdf of X is derived from the transformation of Y as f X (x) = β(λ,λ 2 ) ( x μ )( λ2 ) { +( x μ where β(λ,λ 2 )= Γ(λ )Γ(λ 2 ) Γ(λ + λ 2 ), and Γ() = 0 t e t dt. There is also another definition for the FP distribution. ) } (λ +λ 2 ), x μ, (2) Definition 2.2: Let Y and Y 2 be two independent random variables having gamma distribution with scale parameter and shape parameters λ and λ 2.Then X = μ + has Y2 Y FP(μ,,, λ,λ 2 ) distribution, where f X (x) is defined in (2). Arnold [, chap.3] showed that the FP distribution is a generalization of the Pareto (IV) distribution, so the density function of the Pareto (IV) (μ,,, ) is given by g X (x) = x μ x μ + (+), x μ, (3) where <μ<+ is the location parameter, >0 is the scale parameter, > 0is the inequality parameter and > 0 is the shape parameter which characterizes the tail of the distribution.also, Klugman et al.[6] showed that the density function of the transformed beta(tb) family is given by t X (x) = x τ { x } (+τ) +, x > 0. (4) β(, τ)x Moreover, we can see in the following summary, these families are still quite general and include other important distributions.

3 Shannon entropy for the Feller-Pareto (FP) family 497 Pareto (IV)(μ,,, ) =FP(μ,,,, ) : TB(,,, τ) =FP(0,,,,τ): Pareto (I)(, ) = FP(,,,,), Pareto (II)(μ,, ) = FP(μ,,,,), Pareto (III)(μ,, ) = FP(μ,,,, ), Burr(,, ) = TB(,,, ), loglogistic(, ) = Burr(,, ), Paralogistic(, ) = Burr(,, ), Generalized Pareto (,, τ) =TB(,,,τ): Scaled-F distribution(, τ) = Generalized Pareto(,,τ), Inverse Pareto(II)(, τ) = Generalized Pareto(,, τ), Inverse Burr (,, τ) =TB(,,, τ): loglogistic(, ) = Inverse Burr(,, ), Inverse Pareto(, τ) = Inverse Burr(,, τ), Inverse Paralogistic(, ) = Inverse Burr(,, ). Now, Suppose X is a random variable with FP(μ,,, λ,λ 2 ) distribution with pdf of Feller- Pareto given in (2). Formula () is important for our computations. The log-density of (2) is λ2 x μ log f X (x) = log (β(λ,λ 2 )) log()+ log x μ (λ + λ 2 ) log +, (5) and the entropy is Γ(λ + λ 2 ) H(X) = E ( log f X (x)) = log log(γ(λ )) log(γ(λ 2 )) ( + λ 2 ) ( E log To determine an expression for H(X), we need to find E ) X μ +(λ + λ 2 )E log X μ +. (6) ( log ( X μ Derivation of these expectations are based on the following strategy. )) and E log X μ +. h(r) =E[( X μ ) r ] = μ β(λ,λ 2 ) (x μ λ ) r+ 2 {+( x μ ) } (λ +λ 2 ) dx. (7)

4 498 S. Tahmasebi and J. Behboodian x μ u By the change of variable =, 0 <u<, we obtain: u [ X μ r ] h(r) = E = 0 β(λ,λ 2 ) ur+λ 2 ( u) λ r du = Γ(r + λ 2)Γ(λ r), λ r 0,, 2,... (8) Γ(λ )Γ(λ 2 ) Differentiating both side of (8) with respect to r we obtain: d dr μ h(r) = E[(X = ) r log( X μ )] Γ(λ )Γ(λ 2 ) [Γ (r + λ 2 )Γ(λ r) Γ(r + λ 2 )Γ (λ r)]. (9) From relation (9), at r = 0 we obtain [ ] X μ [ E log = Γ (λ 2 )Γ(λ ) Γ(λ 2 )Γ (λ ) ] Γ(λ )Γ(λ 2 ) = [ψ(λ 2 ) ψ(λ )], (0) where ψ is the digamma function defined by ψ(λ) = d log Γ(λ). dλ Now we calculate r k(r) = E X μ + = μ β(λ,λ 2 ) λ 2 x μ x μ By using the change of variable = { +( x μ t, 0 <t<, we obtain: t ) } (λ +λ 2 )+r dx. () k(r) = t λ2 ( t) λ r dt β(λ,λ 2 ) 0 = Γ(λ + λ 2 )Γ(λ r) Γ(λ )Γ(λ + λ 2 r). (2) r k (r) = E X μ + log X μ + = Γ(λ + λ 2 )[Γ(λ r)γ (λ + λ 2 r) Γ (λ r)γ(λ + λ 2 r)] Γ(λ )[Γ(λ + λ 2 r)] 2. (3)

5 Shannon entropy for the Feller-Pareto (FP) family 499 Using relation (3), at r = 0 we have E log X μ + = Γ(λ + λ 2 ) [Γ(λ )Γ (λ + λ 2 ) Γ (λ )Γ(λ + λ 2 ] Γ(λ )[Γ(λ + λ 2 ] 2 = ψ(λ + λ 2 ) ψ(λ ). (4) Putting (0) and (4) in relation (6) we have: H(X) = Γ(λ + λ 2 ) log log(γ(λ )) log(γ(λ 2 )) +( λ 2 )[ψ(λ 2 ) ψ(λ )] +(λ + λ 2 )[ψ(λ + λ 2 ) ψ(λ )]. (5) We note that Shannon entropy for other related distributions are given in Table. Table : Table of Shannon Entropy family name density function Shannon entropy Pareto (IV)(μ,,, ) Pareto (III)(μ,, ) Pareto (II)(μ,, ) Pareto (I)(, ) Transformed Beta(,,, τ) x μ f X (x) ( ) + x μ + x > μ, μ (, ) >0, >0, >0 f X (x) = x μ { } 2 x μ + x > μ, μ R, > 0, >0 f X (x) = ( ( x μ + x>μ,>0, > 0 f X (x) = ( ( x + x>,>0, > 0 Γ( + τ) x τ f X (x) = Γ()Γ(τ)x ( x ) +τ + x>0, >0,>0, τ>0, > 0 )) + )) + H(X) = log +( )(ψ() ψ()) + + H(X) = log ()+2 ( H(X) = log + ) + ( H(X) = log + ) + = H(Pareto (II)) H(X) = log(β(, τ)) + log + τ [ψ(τ) ψ()] +( + τ)[ψ( + τ) ψ()]

6 500 S. Tahmasebi and J. Behboodian family name density function Shannon entropy f Burr(,, ) X (x) = x ( x ) (+) H(X) = log + x>0, >0,>0, > 0 +( + )(ψ() ψ()) + f loglogistic(, ) X (x) = x ( x ) 2 + H(X) = log +2 x>0, >0, > 0 Scaled-F distribution f X (x) = β(, τ) xτ ( + x) (+τ) H(X) = log(β(, τ)) +( τ)[ψ(τ) ψ()] (, τ) x>0,, > 0, τ > 0 +( + τ)[ψ( + τ) ψ()] f Paralogistic(, ) X (x) = 2 x ( x ) (+) H(X) = log 2 + +( )(ψ() ψ()) Generalized Pareto (,, τ) Inverse Pareto (, τ) Inverse Burr (,, τ) Inverse Paralogistic (, ) x>0, > 0,>0 Γ( + τ) x τ f X (x) = ( Γ()Γ(τ)x + x ) +τ x>0, > 0, > 0, τ > 0 f X (x) = τ ( x τ ( + x ) ) (+τ) x x>0, > 0, τ > 0 f X (x) = τ ( x τ ( x ) ) (+τ) + x x>0, >0, τ > 0, > 0 f X (x) = ( x ( x ) ) 2 + x x>0, > 0, > 0 3. The Shannon Entropy for Order statistics of FP Subfamilies: + + H(X) = log(β(, τ)) + log () +( τ)[ψ(τ) ψ()] +( + τ)[ψ( + τ) ψ()] H(X) = log (τ) +( τ)[ψ(τ) ψ()] +( + τ)[ψ( + τ) ψ()] + ( τ ) H(X) = log τ [ψ(τ) ψ()] +( + τ)[ψ( + τ) ψ()] H(X) = log +2 [ψ(2) ψ()] Let X,..., X n be a random sample from a distribution F X (x) with density f X (x) > 0. The order statistics of this sample is defined by the arrangement of X,..., X n from the smallest to the largest, by Y <Y 2 <... < Y n. The density of Y j, j =,..., n, is n! f Yj (y) = (j )!(n j)! f X(y)[F X (y)] j [ F X (y)] n j. (6) Now, let U,U 2,..., U n be a random sample from U(0, ) with the order statistics W <W 2 <... < W n. The density of W j, j =,..., n, is f Wj (w) = B(j, n j +) wj [ w] n j, 0 <w<, (7)

7 Γ(j)Γ(n j +) where B(j, n j +)= = Γ(n +) The entropy of the beta distribution is Shannon entropy for the Feller-Pareto (FP) family 50 (j )!(n j)!. n! H n (W j )= (j ) [ψ(j) ψ(n + )] (n j)[ψ(n + j) ψ(n + )] + log B(j, n j +), d log Γ(t) where ψ(t) =,ψ(n +)=ψ(n)+ dt n. Using the fact that W j = F X (Y j ) and Y j = FX (W j), j=, 2,..., n, are one to one transformations, the entropies of order statistics can be computed by [ ] H(Y j ) = H n (W j ) E gj log f X (FX (W j)) = H n (W j ) f j (y) log f X (y)dy (8) where H n (W j ) is the entropy and E gj is an expectation of beta distribution. Now, we can have an application of (8) for the pareto (IV) distribution. Let X be a random variable having x μ Pareto type (IV) (μ,,, ) with F X (x) = +. For computing H(Y j ), we find FX (W j)= ) [(( W j ) ] + μ and the expectation term in (8) as follows: [ ] E gj log f X FX (W j) j = log j n! +( ). (n j)! k ψ() ψ((n j + k + )) ( ) k!(j k )!(n j + k +) k=0 + + (ψ(n j +) ψ(n + )) ( ) = log +( )t,j (n) + + (ψ(n j +) ψ(n + )), (9) n! k ψ() ψ((n j + k + )) where t,j (n) = ( ) (n j)! k!(j k )!(n j + k +). k=0 Now, by (8) and (9) the entropy of order statistics is H(Y j )=H n (W j ) log +( +)t,j (n)+ + (ψ(n)+ n ) ψ(n j +). (20) In particular cases, the entropy for the sample minimum and maximum are H(Y )= log n n + +( +)(ψ() ψ(n)) + n, (2)

8 502 S. Tahmasebi and J. Behboodian and n n n H(Y n ) = log +( +) ( ) k (ψ() ψ((k + ))) k + k=0 + n n + + (ψ(n +) ψ()). (22) If n =2m + and 0 <, then the difference between H(Y 2m+ ) and H(Y )is ) 2m ( ) k=0 ( ) k ( 2m + k + [ψ((k + )) ψ()] + + (ψ(2m +) ψ()) 0. (23) Thus, when n is odd,the entropy about the maximum is more than the entropy about the minimum in Pareto (IV) samples. Finally, we present Shannon entropy of order statistic and entropy ordering property for the sample minimum and maximum of several FP subfamilies in Table 2.

9 Shannon entropy for the Feller-Pareto (FP) family 503 Table 2: Entropy of order statistics and entropy ordering for FP subfamilies FP subfamilies and Distribution function Entropy for order statistics Entropy ordering Pareto (III)(μ,, ) H(Y j )=H n (W j ) + log () x μ F X (x) = + H(Y +( )t,j (n) ) H(Y n ) for n =2m + +2 (ψ(n +) ψ(n j + )) x>μ, >0, 0 < Pareto (II)(μ,, ) ( ( x μ F X (x) = + x>μ,>0, > 0 Pareto (I)(, ) ( ( x F X (x) = + x>,>0, > 0 Transformed Beta(,,, ) ( x ) F X (x) = + )) )) x>0, > 0,>0, 0 < Loglogistic(, ) ( x ) F X (x) = + x>0, > 0, 0 < Paralogistic(, ) ( x ) F X (x) = + x>0, > 0, 0 < Generalized Pareto(,, ) ( F X (x) = + x ) x>0, > 0, > 0 Inverse Pareto(, ) ( F X (x) = + x ) x>0, > 0 Inverse Burr(, ) F X (x) = ( + x) x>0, > 0 H(Y j )=H n (W j ) + log H(Y ) H(Y n ) + + (ψ(n +) ψ(n j + )) for all n H(Y j )=H n (W j ) + log H(Y ) H(Y n ) + + (ψ(n +) ψ(n j + )) for all n H(Y j )=H n (W j ) log ( ) + t,j(n) (ψ(n +) ψ(n j + )) + + H(Y j )=H n (W j ) + log () + t,j(n) +2 (ψ(n +) ψ(n j + )) H(Y j )=H n (W j ) 2 log () + log()+ t,j(n) (ψ(n +) ψ(n j + )) + + H(Y ) H(Y n ) for n =2m + H(Y ) H(Y n ) for n =2m + H(Y ) H(Y n ) for n =2m + H(Y j )=H n (W j ) + log( ) H(Y ) H(Y n ) + + (ψ(n +) ψ(n j + )) for all n H(Y j )=H n (W j ) + log() +2 (ψ(n +) ψ(n j + )) H(Y j )=H n (W j ) log() +2 (ψ(n +) ψ(n j + )) H(Y ) H(Y n ) for all n H(Y ) H(Y n ) for all n

10 504 S. Tahmasebi and J. Behboodian Conclusion We have derived the exact form of Shannon entropy for the Feller-Pareto(FP) family and order statistics of FP Subfamilies.These families cover a wide spectrum of areas ranging from actuarial science,economics, financial to medicine and telecommunications, for distributions of important variables such as sizes of insurance claims, incomes in a population of people, stock price fluctuations, and length of telephone calls. So we believe that the results that we have presented in this paper will be important as a reference for scientists and engineers from many areas. References [] B.C. Arnold, Pareto Distributions, Internationl cooperative, Publishing House, Fairland, MD(983). [2] B.C. Arnold, L. Laguna, On Generalized Pareto Distributions with Applications to Income Data. International Studies in economics, Monograph#0, Deparetment of Economics, Iowa State University, 977. [3] T.M. Cover, J.A. Thomas, Elements of information theory, Wiley, New York, 99. [4] W. Feller, An Introduction to probability Theory and its Applications, Vol 2, 2nd Edition, Wiley, New York, 97. [5] J.N. Kapur, Measure of information and their Applications, Wiley, New York, 994. [6] S. A. Klugman, H. H. Panjer, and G. E. Willmot, Loss models: From Data to Decision, Wiley, New York,998. [7] S. Kullback, Information and Statistics, Wiley, New York, 959. [8] A.C. Lazo, P.N. Rathie, On the entropy of continuous Probability distributions, IEEE Transactions of information Theory, 24(978), [9] S. Park, The entropy of consecutive order statistics, IEEE Transactions of information Theory, 4(995), [0] A. Renyi, On measures of entropy and information, Proc. 4th Berkeley Symposium, Stat. Probability, (96), [] C.E. Shannon, A mathematical theory of communication, Bell System Technical Journal, 27(948), [2] K.M. Wong, S. Chen, The entropy of ordered sequances and order statistics, IEEE Transactions of information Theory, 36(990), Received: June, 2009