Preprint
Article

An Intrinsic Characterization of Shannon’s and Rényi’s Entropy

Altmetrics

Downloads

2

Views

2

Comments

0

This version is not peer-reviewed

Submitted:

15 November 2024

Posted:

19 November 2024

You are already at the latest version

Alerts
Abstract
All characterizations of Shannon’s entropy include the so-called chain rule, a formula on a hierarchically structured probability distribution, which is based on at least two elementary distributions. We show that the chain rule can be split into two natural components, the well-known additivity of the entropy in case of cross-products and a variant of the chain rule that involves only a single elementary distribution. The latter is given as a proportionality relation and hence allows a vague interpretation as self-similarity, hence intrinsic property of the Shannon entropy. A similar characterization is given also for the Rényi entropy and the min-entropy.
Keywords: 
Subject: Computer Science and Mathematics  -   Applied Mathematics
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated