Mutual Information and Nonadditive Entropies: The Case of Tsallis Entropy

Author(s): Amelia Carolina Sparavigna

Mutual information of two random variables can be easily obtained from their Shannon entropies. However, when nonadditive entropies are involved, the calculus of the mutual information is more complex. Here we discuss the basic matter about information from Shannon entropy. Then we analyse the case of the generalized nonadditive Tsallis entropy.

Mutual Information, Entropy, Tsallis Entropy, Generalized Additivity, Image Registration

1. J.W. Kay, D. M. Titterington (1999). Statistics and Neural Networks: Advances at the Interface, Oxford University Press. ISBN-13: 978-0198524229, ISBN-10: 0198524226
2. R. W. Hamming (1980). Coding and Information Theory, Prentice-Hall. ISBN-13: 978-0521196819, ISBN-10: 0521196817
3. S. Furuichi (2006). Information Theoretical Properties of Tsallis Entropies, J. Math. Phys. 47:023302. DOI:10.1063/1.2165744
4. C. Tsallis (1988). Possible Generalization of Boltzmann-Gibbs Statistics, Journal of Statistical Physics 52:479. DOI:10.1007/BF01016429
5. A.C. Sparavigna (2015). Shannon, Tsallis and Kaniadakis Entropies in Bi-Level Image Thresholding, Int. J. Sci. 4(2):35. DOI: 10.18483/ijSci.626
6. S. Martin, G. Morison, W. Nailon, T. Durrani (2004). Fast and Accurate Image Registration Using Tsallis Entropy and Simultaneous Perturbation Stochastic Approximation, Electronics Letters, 40(10):595. DOI: 10.1049/el:20040375
7. Yue Deng (2014). High-Dimensional and Low-Quality Visual Information Processing, Springer. ISBN: 978-3-662-44526-6
8. V. Christophides, V. Efthymiou, K, Stefanidis (2015). Entity Resolution in the Web of Data, Morgan & Claypool Publishers. DOI: 10.2200/S00655ED1V01Y201507WBE013
9. Vv. Aa., Mutual Information, Wikipedia, Retrieved 3 October 2015.
10. G.J. Pottie, W.J. Kaiser (2005). Principles of Embedded Networked Systems Design, Cambridge University Press. Online ISBN: 9780511541049, DOI: 10.1017/CBO9780511541049
11. T.M. Cover, J.A. Thomas (1991). Elements of Information Theory, John Wiley & Sons, Inc. Print ISBN 0-471-06259-6, Online ISBN 0-471-20061-1
12. Vv. Aa., Conditional Entropy, Wikipedia, Retrieved 3 October 2015.
13. A. Rényi (1960). On Measures of Information and Entropy, Proceedings of the Fourth Berkeley Symposium on Mathematics, Statistics and Probability, pp. 547–561.
14. S. Abe, A.K. Rajagopal (2000). Nonadditive Conditional Entropy and its Significance for Local Realism, arXiv:quant-ph/0001085, 24 Jan 2000.
15. J.P.W. Pluim, J.B. Antoine Maintz, M.A. Viergever (2003). Mutual-Information-Based Registration of Medical Images: A Survey, IEEE Transactions on Medical Imaging, 22(8):986. DOI: 10.1109/tmi.2003.815867
16. A.C. Sparavigna (2015). Conditional Kaniadakis Entropy: a Preliminary Discussion. PHILICA.COM Article number 524.