Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter September 30, 2022

Estimation of entropy and extropy based on right censored data: A Bayesian non-parametric approach

  • Luai Al-Labadi EMAIL logo and Muhammad Tahir

Abstract

Entropy and extropy are central measures in information theory. In this paper, Bayesian non-parametric estimators to entropy and extropy with possibly right censored data are proposed. The approach uses the beta-Stacy process and the difference operator. Examples are presented to illustrate the performance of the estimators.

MSC 2010: 94A17; 62F15

References

[1] L. Al-Labadi and S. Berry, Bayesian estimation of extropy and goodness of fit tests, J. Appl. Stat. 49 (2022), no. 2, 357–370. 10.1080/02664763.2020.1812545Search in Google Scholar PubMed PubMed Central

[2] L. Al-Labadi and M. Evans, Prior-based model checking, Canad. J. Statist. 46 (2018), no. 3, 380–398. 10.1002/cjs.11457Search in Google Scholar

[3] L. Al-Labadi, V. Patel, K. Vakiloroayaei and C. Wan, A Bayesian nonparametric estimation to entropy, Braz. J. Probab. Stat. 35 (2021), no. 2, 421–434. 10.1214/20-BJPS483Search in Google Scholar

[4] L. Al-Labadi, V. Patel, K. Vakiloroayaei and C. Wan, Kullback–Leibler divergence for Bayesian nonparametric model checking, J. Korean Statist. Soc. 50 (2021), no. 1, 272–289. 10.1007/s42952-020-00072-7Search in Google Scholar

[5] L. Al Labadi and M. Zarepour, A Bayesian nonparametric goodness of fit test for right censored data based on approximate samples from the beta-Stacy process, Canad. J. Statist. 41 (2013), no. 3, 466–487. 10.1002/cjs.11188Search in Google Scholar

[6] A. I. Al-Omari, A new measure of entropy of continuous random variable, J. Stat. Theory Pract. 10 (2016), no. 4, 721–735. 10.1080/15598608.2016.1217444Search in Google Scholar

[7] H. Alizadeh Noughabi, A new estimator of entropy and its application in testing normality, J. Stat. Comput. Simul. 80 (2010), no. 9–10, 1151–1162. 10.1080/00949650903005656Search in Google Scholar

[8] D. Applebaum, Lévy Processes and Stochastic Calculus, Cambridge Stud. Adv. Math. 93, Cambridge University, Cambridge, 2004. 10.1017/CBO9780511755323Search in Google Scholar

[9] J. Beirlant, E. J. Dudewicz, L. Györfi and E. C. van der Meulen, Nonparametric entropy estimation: An overview, Int. J. Math. Stat. Sci. 6 (1997), no. 1, 17–39. Search in Google Scholar

[10] J. C. Correa, A new estimator of entropy, Comm. Statist. Theory Methods 24 (1995), no. 10, 2439–2449. 10.1080/03610929508831626Search in Google Scholar

[11] T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., John Wiley & Sons, New York, 2006. Search in Google Scholar

[12] R. B. D’Agostino and M. A. Stephens, Goodness-of-Fit Techniques, Stat. Textb. Monogr. 68, Marcel Dekker, New York, 1986. Search in Google Scholar

[13] N. Ebrahimi, K. Pflughoeft and E. S. Soofi, Two measures of sample entropy, Statist. Probab. Lett. 20 (1994), no. 3, 225–234. 10.1016/0167-7152(94)90046-9Search in Google Scholar

[14] T. S. Ferguson, A Bayesian analysis of some nonparametric problems, Ann. Statist. 1 (1973), 209–230. 10.1214/aos/1176342360Search in Google Scholar

[15] T. S. Ferguson and E. G. Phadia, Bayesian nonparametric estimation based on censored data, Ann. Statist. 7 (1979), no. 1, 163–186. 10.1214/aos/1176344562Search in Google Scholar

[16] P. A. Grzegorzewski and R. Wieczorkowski, Entropy-based goodness-of-fit test for exponentiality, Comm. Statist. Theory Methods 28 (1999), no. 5, 1183–1202. 10.1080/03610929908832351Search in Google Scholar

[17] R. Hazeb, M. Z. Raqab and H. A. Bayoud, Non-parametric estimation of the extropy and the entropy measures based on progressive type-II censored data with testing uniformity, J. Stat. Comput. Simul. 91 (2021), no. 11, 2178–2210. 10.1080/00949655.2021.1888953Search in Google Scholar

[18] D. Källberg and O. Seleznjev, Estimation of entropy-type integral functionals, Comm. Statist. Theory Methods 45 (2016), no. 4, 887–905. 10.1080/03610926.2013.853789Search in Google Scholar

[19] Y. Kim and J. Lee, On posterior consistency of survival models, Ann. Statist. 29 (2001), no. 3, 666–686. 10.1214/aos/1009210685Search in Google Scholar

[20] F. Lad, G. Sanfilippo and G. Agrò, Extropy: Complementary dual of entropy, Statist. Sci. 30 (2015), no. 1, 40–58. 10.1214/14-STS430Search in Google Scholar

[21] J. F. Lawless, Statistical Models and Methods for Lifetime Data, 2nd ed., Wiley Ser. Probab. Math. Statist., John Wiley & Sons, New York, 2003. 10.1002/9781118033005Search in Google Scholar

[22] J. Lee and Y. Kim, A new algorithm to generate beta processes, Comput. Statist. Data Anal. 47 (2004), no. 3, 441–453. 10.1016/j.csda.2003.12.008Search in Google Scholar

[23] T. A. Mazzuchi, E. S. Soofi and R. Soyer, Bayes estimate and inference for entropy and information index of fit, Econometric Rev. 27 (2008), no. 4–6, 428–456. 10.1080/07474930801960311Search in Google Scholar

[24] H. A. Noughabi and N. R. Arghami, A new estimator of entropy, J. Iran. Stat. Soc. (JIRSS) 9 (2010), no. 1, 53–64. Search in Google Scholar

[25] H. A. Noughabi and J. Jarrahiferiz, Extropy of order statistics applied to testing symmetry, Comm. Statist. Simulation Comput. 51 (2022), no. 6, 3389–3399. 10.1080/03610918.2020.1714660Search in Google Scholar

[26] S. Park, Testing exponentiality based on the Kullback–Leibler information with the type II censored data, IEEE Trans. Reliab. 54 (2005), 22–26. 10.1109/TR.2004.837314Search in Google Scholar

[27] E. G. Phadia, Prior Processes and Their Applications, Springer, Heidelberg, 2013. 10.1007/978-3-642-39280-1Search in Google Scholar

[28] G. Qiu and K. Jia, Extropy estimators with applications in testing uniformity, J. Nonparametr. Stat. 30 (2018), no. 1, 182–196. 10.1080/10485252.2017.1404063Search in Google Scholar

[29] A. Rényi, On measures of entropy and information, Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, University of California, Berkeley (1961), 547–561. Search in Google Scholar

[30] C. E. Shannon, A mathematical theory of communication, Bell System Tech. J. 27 (1948), 379–423, 623–656. 10.1002/j.1538-7305.1948.tb01338.xSearch in Google Scholar

[31] B. van Es, Estimating functionals related to a density by a class of statistics based on spacings, Scand. J. Stat. 19 (1992), no. 1, 61–72. Search in Google Scholar

[32] O. Vasicek, A test for normality based on sample entropy, J. Roy. Statist. Soc. Ser. B 38 (1976), no. 1, 54–59. 10.1002/9781119186229.ch35Search in Google Scholar

[33] S. Walker and P. Muliere, Beta-Stacy processes and a generalization of the Pólya-urn scheme, Ann. Statist. 25 (1997), no. 4, 1762–1780. 10.1214/aos/1031594741Search in Google Scholar

[34] R. Wieczorkowski and P. A. Grzegorzewski, Entropy estimators—improvements and comparisons, Comm. Statist. Simulation Comput. 28 (1999), no. 2, 541–567. 10.1080/03610919908813564Search in Google Scholar

Received: 2022-04-18
Revised: 2022-09-12
Accepted: 2022-09-13
Published Online: 2022-09-30
Published in Print: 2022-12-01

© 2022 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 28.3.2024 from https://www.degruyter.com/document/doi/10.1515/mcma-2022-2123/html
Scroll to top button