Probability has always been extraordinarily difficult to tie down. Edwin Jaynes made fundamental contributions to the subject, and in his magisterial monograph (published posthumously: “Probability Theory: The Logic of Science”[1]) spends two chapters on its principles and elementary applications, commenting on its ‘weird’ and ‘complicated’ history. Indeed, Jaynes’ motivation was aimed at helping the interested reader who already has “a previous acquaintance with probability and statistics” to essentially “unlearn” much of what they may have previously learned!

The ubiquity (and longevity) of fallacies in the use of statistics indicates its difficulty (on the misuse of the “p-value” see for example Goodman 2008[2] and Halsey 2019[3]; on the persistence of fallacies see Smaldino & McElreath 2016[4]). A significant part of the problem may be related to the fact that the “probability” of an event is apparently not a property solely of “external reality”: since there must be someone assessing the probability, it must be a function of not only what information already exists about the event but also who knows. The fact that our estimates of the probability of some event invariably involve our prior knowledge, combined with the fact that all knowledge ultimately involves the properties of recursive statements (by Gödel’s Theorem[5]; this point has recently been elaborated by Jeynes et al. [6]) mean that some part of this difficulty must be due to the neglecting, in current (simplified) treatments, of recursive dependencies (such as the “Chicken and Egg” problem, on which see §2.3 below).

In 1946 R.T.Cox observed, acutely, that the concept of probability mixes two separate concepts (“the idea of frequency in an ensemble and the idea of reasonable expectation”[7]) which are now represented by two schools usually called the “frequentists” and the “Bayesians”. We point out that Enßlin et al. [8] comment that, “The different views on probabilities of frequentists and Bayesians are addressed by … Allen Caldwell[9] [who] builds a bridge between these antagonistic camps by explaining frequentists’ constructions in a Bayesian language”, and we will assume that Caldwell is right and therefore that we can ignore the philosophical distinction between the frequentists and the Bayesians.

Building on Cox’s fundamental work we will here derive a rigorous treatment of recursive probability which is of general applicability. In particular, we will treat probability as a physical quantity grounded in hyperbolic Minkowski spacetime, and obeying all the appropriate physical laws. The close relation of probability to the new Quantitative Geometrical Thermodynamics[10] (QGT: this constructs the quantity info-entropy by treating information and entropy as Hodge duals) is due to the fact that the (hyperbolic) entropic velocity q′≡dq/dx in QGT is dimensionless and has properties akin to a probability (0≤q′≤1); noting that q′ is isomorphic to the more familiar kinematic velocity ẋ ≡ dx/dt (0≤ ẋ ≤c, where c is the speed of light) in hyperbolic (Minkowski) spacetime (see ref.[10]), and which also obeys the well-known velocity addition theorem of Special Relativity.

The treatment of probability as a physical quantity (and obeying physical laws) is as fundamental a concept as the well-known statement by Rolf Landauer that “Information is Physical”[11].

We are used to treating both probability and information anthropomorphically (that is, depending on what you or I might be expected to know): here we will establish an impersonal sense for probability, in the same way that Landauer insisted on the (impersonal) Shannon entropy sense of his Information. Note that each of information and entropy both use and require probabilistic quantities in their fundamental definitions; and also note that the consequences of treating information as being a quantity as physical as is energy (for example) has led to important insights, and not only into basic engineering problems of the global internet (for an example of which see Parker et al. 2015[12]). We expect similar advances to follow from recognising probability as also being an equally physical quantity.

The relationship between probability and the quantification of information using the Shannon entropy is well understood mathematically, but its interpretation as a physical theory has been convoluted. Although the Shannon entropy was quickly recognised as important, Edwin Jaynes’ formulation of Maximum Entropy (MaxEnt) theory, where the Shannon metric plays a key role, was initially controversial and took some decades to achieve acceptance (see Jaynes’ 1978 summary[13]). However, MaxEnt as a powerful scientific and engineering tool has helped considerably to underpin the physicality of information, and therefore also acts as a support to the underlying assertion of this paper (paraphrasing Landauer) that “Probability is Physical”. We will also explore the implications of this assertion.

Since the concept of MaxEnt will be centrally important in this work, it is worth adding Jaynes’ authoritative definition (1982)[14]: “The MaxEnt principle, stated most briefly, is: when we make inferences based on incomplete information, we should draw them from that probability distribution that has the maximum entropy permitted by the information we do have”.

Torsten Enßlin’s treatment[15] of information as a field is interesting in this context: he considers that a “physical field has an infinite number of degrees of freedom since it has a field value at each location of a continuous space” where he is specifically considering imaging problems of astrophysical datasets. Enßlin et al. [ref.8] treat information informally as an anthropomorphic concept: to the question, “What is information?” they answer “Anything that changes our minds and states of thinking!” But here we will treat information (and probability) as physical quantities, not as anthropomorphic concepts; and especially noting that infinite quantities are inimical to physical phenomena. In particular, in our QGT treatment, information is formally defined as a physical quantity (albeit in terms making full use of the properties of analytical continuation, which is itself closely related to fundamental physical precepts such as causality[16] and square-integrability ensuring finite physical quantities[17]), so that the number of degrees of freedom are finite and may be very small, as is observed for the geometrical entropy of isotopes of the helium nucleus[18]. Such results for information were already pointed out by Parker & Walker (2004)[19] who investigated the residues of a meromorphic function (that is, a function analytic nearly everywhere) due to the presence of isolated singularities in the complex plane (singularities which are entirely analogous to particles in their behaviour); and show that the information (entropy) of such a function is simply given by the sum of the residues.

This is immediately applicable to the Schrödinger equation: it is interesting that we will conclude (Eq.7) that the appropriate Sum Rule for recursive probabilities has a hyperbolic form, and we will draw out the relation of this to the (hyperbolic) entropic velocities in the QGT formalism[20], in which the entropic Uncertainty Principle and entropic isomorphs of the Schrödinger equation may be derived from the entropic Liouville Theorem; all based on the Boltzmann constant as the relevant quantum of entropy. Of course, QGT is constructed in a hyperbolic (complex) Minkowski 4-space (see §18.4 in Roger Penrose’s “Road to Reality”[21]; as another pertinent example, Maxwell’s electro-magnetic field is a hyperbolic version of the Cauchy-Riemann Equations: see Courant & Hilbert[22] vol.II ch.III §2 Eq.8 passim).

The ramifications of this are very wide. Parker & Jeynes [ref.10] have already shown the relevance of QGT to the stability and structure of spiral galaxies (using the properties of black holes), and also that entropy production (dS/dt) is conserved even in relativistic Maximum Entropy systems[23]. These issues up to now have been treated as problems in quantum gravity, and Matt Visser[24] very helpfully reviews conservative entropic forces (in Euclidean space and non-relativistically, although he comments that representing general relativity entropically should be possible). Note also that Visser suggests that the negative entropies that appear in his treatment can be regarded as information, citing Brillouin’s idea of “negentropy” which Parker & Jeynes [ref.10] have shown to be a subtle misapprehension of information and entropy (which are actually Hodge duals).

We believe that much progress may be made by using the coherent formalism of QGT (defined in hyperbolic space) which has been shown to apply to both quantum mechanical and gravitational systems (that is, at all scales from sub-atomic to cosmic: see [refs.18,10]), and since quantum mechanics is built on probabilities, the demonstration here that the general recursive sum rule for probabilities is hyperbolic is a significant conceptual regularisation. This conclusion is reinforced by Knuth’s demonstration[25] that: “The sum and product rules, which are familiar from, but not unique to, probability theory, arise from the fact that logical statements form a distributive (Boolean) lattice, which exhibits the requisite symmetries”. Moreover, Jaeger[26] reviews a variety of treatments, some of which involve theories of generalized probability, aimed at deriving quantum mechanics from information theory.

The basic isomorphism for the hyperbolic sum rule for probabilities (that we will prove, see Eq.7) is the (purely mathematical) double-angle identity for the hyperbolic tangent function:

Another interesting and very simple isomorphism is the well-known relativistic sum rule for velocities {u, v}, given by Jackson[27] in his well-known textbook (in the context of a discussion of aberration and the Fizeau experiment; §11.4, Eq.11.28): where c is the speed of light. Jackson comments that if u = c then also w = c, which is an “explicit statement of Einstein’s second postulate” (c is a constant).

This latter (Eq.1b) is clearly physical (since c is involved) where the former (Eq.1a) is a mathematical identity. Note also that in optics the basic formula for the two-layer Fabry-Perot cavity (etalon) is well-known[28]: where the overall scattering (reflectivity) coefficient r₃ is due to a pair of sequential Fresnel reflections (r₁, r₂) separated by a distance Δz for a light ray of propagation constant k; and where we note that light is the physical phenomenon par excellence exhibiting the physics of Special Relativity within the context of hyperbolic (Minkowski) spacetime. Corzine et al.[29] demonstrate that this formula is closely related to the hyperbolic addition rule (Eq.1a) specifically by using a hyperbolic tangent substitution which dramatically simplifies the use of the formula in real (multilayer) cases – see Appendix F for additional and related discussion.

This approach has recently been supported in an interesting way by Skilling & Knuth[30], who conclude: “it must be acknowledged, quantum theory works. So does probability. And the two are entirely mutually consistent.” Their argument shows logical reasons why probability should be regarded as physical.

Skilling & Knuth claim not to be interested (for these purposes) in the distinction between ontology and epistemology. They say (ibid. §3.5):

The ontology–epistemology divide is, for quantitation at least, a distinction without a difference. A bit of information carries no flag to inform us whether it was assigned by a conscious agent or by a mechanical switch. Our job in science is to make sense of our observations, not to indulge in empty disputation between isomorphic views. Our goal here is derivation of a calculus fit for general purpose. Ontology and epistemology share the same symmetries, the same rules, and the same assignments. So they share a common calculus.

which is suggestive of Karen Barad’s (2007[31]) insistence that the distinction between ontology and epistemology is not a real one, and therefore speaking strictly we should refer to “onto-epistemology” (ibid. p.43). Skilling & Knuth also say (ibid. §1):

But, if our object can perturb a partner object, then by symmetry the partner object can also perturb our object. We could assign either role to either.

Our calculus, whatever it is, must be capable of representing such interactions … This insight that interactions are basic is the source of “quantum-ness.”

Again, this recalls Barad’s thesis that “the primary ontological unit is the phenomenon” (ibid. p.333). However, when Skilling & Knuth (ibid. §4) say, “We start with an identifiable object”, this is directly contradicted by Barad, who asserts that “objects” do not have “an inherent ontological separability” (ibid. p.340); that is, strictly speaking, identifiable objects do not actually exist per se (since everything is entangled with everything else). But Skilling & Knuth are not aiming at philosophical precision, only at a demonstrable computability; for these purposes such fine distinctions do not matter. They are right to avoid metaphysical considerations in scientific work: although when wider social implications are important it may be necessary to consider the metaphysics (see for example [ref.6]).

However, it turns out that the inescapable human dimension appears to be especially pronounced in probability, in the sense that the very idea of a probability entails one’s personal state of knowledge or ignorance (and Michael Polanyi insisted long ago that all knowledge is necessarily personal[32]). Howson & Urbach[33] have carefully explained why, although Bayesian (and Maximum Entropy) methods are fully (and helpfully) rational: “there seems to be no way of ‘objectively’ defining prior probabilities … this is really no weakness [since] it allows expert opinion due weight, and is a candid admission of the personal element which is there in all scientific work”. Assessing probabilities necessarily entails assessing uncertainties, and this must always involve some value judgments: although we may do our best to speak rationally about such judgments, it cannot be excluded that different people will (rationally) come to different conclusions.

Note that although scientists have a duty to argue rationally, non-scientists also normally behave rationally. Rationality is a property of humans, and only a rather small subset of humans are scientists.

It is necessary to make a few initial simple remarks about Maximum Entropy (MaxEnt) methods, to clarify the discussion. Jaynes said in 1957[34]: “The guiding principle is that the probability distribution over microscopic states which has maximum entropy subject to whatever is known, provides the most unbiased representation of our knowledge of the state of the system. The maximum-entropy distribution is the broadest one compatible with the given information; it assigns positive weight to every possibility that is not ruled out by the initial data.” In our derivation here of the Hyperbolic Sum Rule we emphasise that the proper application of MaxEnt methods precludes the surreptitious introduction of tacit assumptions (“knowledge”). That is, all the “priors” (prior knowledge that conditions the data) must be stated explicitly (and our ignorance implies that some priors must sometimes be estimated, which may involve personal value judgments).

A fully Bayesian analysis requires that all prior knowledge is explicitly stated, including the “knowledge” that there is no knowledge available, in which case an “unbiassed” estimate is required. This is usually stated in terms of the “Principle of Indifference” (PI), but unfortunately there are a set of (“Bertrand”) paradoxes which appear to invalidate the PI. But Parker & Jeynes[35] have resolved these paradoxes using QGT methods by supplying the missing prior information (in the form of the scale invariance condition).

It is essential to handle the “priors” correctly: our Eq.6 below (which is for the recursive case) asserts a hyperbolic relation for the compound “p(A or B | C)” where in general A may depend both on prior conditions C, and also on B (that is, involving the probabilities of A|BC and B|AC). But if {A,B} are recursively dependent, then p(A|BC) simplifies to p(A|C) (because, for recursive {A,B}, p(A|B)= p(A) since B is ultimately dependent on A; and similarly, p(B|A)= p(B) since A is ultimately dependent on B) giving the general relation describing the “Hyperbolic Sum Rule”, HSR (Eq.7).

In the Appendices we explore the Maximum Entropy (MaxEnt) properties of the HSR, including a proof that it really is MaxEnt. We first show how to impose the MaxEnt criterion (Appendix A) using the Partition Function description, and then show (Appendix B) how other simple sum rules may be “MaxEnt” but are otherwise inadmissible. We prove explicitly that the HSR is MaxEnt (Appendix C) and also generalise it for multiple recursive hypotheses (Appendix D). We show that the Conventional Sum Rule (CSR) is also MaxEnt (Appendix E) within its own domain of applicability, and we also generalise the CSR for multiple hypotheses. Both are MaxEnt, that is, neither the HSR nor the CSR ‘smuggle’ in inadvertent or implicit assumptions within their specified contexts; yet, the HSR encompasses a wider domain of physical application that includes recursion between phenomena, whereas the CSR a priori excludes recursion (although there may still be various mutual dependencies). The HSR must also be used where the properties of the recursion are unknown; that is to say, a set of phenomena are known to be correlated, but the mechanism (or ordering) of causation is not known.

Finally in Appendix F, we show the immediate relevance of this treatment to digital signal processing; in particular the handling of “infinite impulse response” and “finite impulse response” filters.

It is important here to point out that the fact that an entity is Maximum Entropy does not mean that the entity has no structure (even though MaxEnt necessarily implies “maximum ignorance”). The reality is more nuanced. For example, we have shown that, at a certain scale, the alpha particle is a (MaxEnt) unitary entity (than which exists no simpler) [ref.18]. But of course, at a different scale we may see the alpha’s constituent four nucleons (protons and neutrons).

Also, being MaxEnt does not preclude change: for example, free neutrons decay (with a half-life calculated ab initio from QGT by Parker & Jeynes 2023[36]). The most extreme known entity that is MaxEnt is the black hole, and black holes necessarily grow (proved by Parker & Jeynes in 2021 [ref.23] and confirmed in 2023[37]). Nor does the fact that some entities are unconditionally stable mean that the Second Law doesn’t apply to them. On the contrary! The matter radius of the alpha is correctly calculated ab initio from QGT [ref.18].

The physical principle of Maximum Entropy embodies the (physical) Second Law of Thermodynamics. Both the CSR and the HSR are MaxEnt, and therefore each also embodies important aspects of the Second Law. Moreover, the probabilities calculated by either Rule (each with its own particular domain of applicability) refer to the probability either of the events themselves or of our reasonable expectations; these are both physical things since both we and they are physical. Either way, our conclusion is underlined that Probability is Physical.

Consider p(A) which is a representation either of “the probability of event A occurring” or, nearly equivalently (logically speaking), “the probability of hypothesis A being true”. Of course, hypotheses and events are entirely different things, but for convenience we will here usually speak of “events” without any loss of generality (ignoring the philosophical frequentist/Bayesian distinction, see §1.1).

The probability that A or B will happen is conventionally given (by the “Conventional Sum Rule”, CSR) as the sum of the separate probabilities that A and B, respectively, will each happen, minus the probability that both of {A,B} will happen together (in general {A,B} are not independent):

This is simply extended to the conditional case treating the probability of A or B given some conditionality C: where A and B are two events, conditional either on each other or some other event(s); in other words, in this case C represents all the possible conditionalities, including p(A|B) etc. Eq.2b represents the CSR. But if {A,B} are recursively dependent (the “chicken and egg” scenario) Eq.2b is invalid since C either cannot be specified explicitly or cannot be specified at all.

Intrinsic to Eqs.2a&b is also the corresponding “Product Rule”, more commonly known as Bayes’ Theorem:

Here, Bayes’ Theorem appears in the latter two expressions of Eq.2c, which yield the same result since Bayes’ Theorem is commutative: p(AB)= p(BA). In particular, Bayes’ Theorem represents a choice as to which of the possible conditionalities should take priority. The logical AND function can be calculated in different ways according to which of the conditionalities is considered to be prior.

We emphasise: although we seek here an alternative form of the Sum Rule (Eq.2b) that takes recursion into account, Bayes’ Theorem (Eq.2c) remains valid.

Such recursive issues arise when we start considering which of the conditionalities is prior (an issue of causation): there appears to be a temporal dependence between the two conditionalities since one must surely occur earlier in time than the other? But note that Special Relativity teaches us that if two spatially-separated events are causally connected (that is, they lie within each other’s light cone) then temporal priority is always preserved whatever the frame of reference (FoR). However, if the events are not causally connected in spacetime then for any FoR where the event of A occurs before that of B, another FoR exists where event B occurs before A. If causal connectedness cannot be determined, either due to the events being spacelike separated, or perhaps because no physical mechanism can be identified to causally connect the events, then there is an ambiguity as to which of the events is prior (and of course, correlation does not imply causation).

An example of a recursive system is a multilayer etalon represented by Eq.1c.

Jaynes showed that the derivation of the product rule (Eq.2c) from the basic laws of Boolean logic is straightforward [ref.1]. However, performing the same derivation for the “Sum Rule” of probability is less obvious. In particular, it is worth noting that Jaynes specifically avoids the use of the Venn diagram as a means to justify Eq.2b, preferring to interpret propositions in terms not of sets but as probability distributions carrying incomplete information. It is useful to note in this context that there is intense current interest in how to handle incomplete (or erroneous) “information” in a formal Bayesian analysis (see for example Zhang et al.[38]).

Jaynes therefore calls the Venn diagram a ‘useful device’ but comments that it can ‘mislead’ [our emphasis]. The Venn diagram is good for representing binary logical operations (see Figure 1) but in general correctly represents neither temporality (including aspects of causality or recursiveness) nor probability distributions. We add that it is also not obvious how the Venn diagram (or the CSR of Eq.2b) could be extended to include multiple hypotheses and/or conditionalities.

Jaynes’ derivation [ref.1] of the “Product Rule” (Eq.2c) does not need to involve de Morgan’s theorems (which follow from the rules of Boolean logic). But his derivation of the “Sum Rule” (Eq.2b) adopts the original schema of Cox [ref.7] to also define an auxiliary function S so that v ≡ S(u), where u ≡ p(A|C) such that v ≡ p(A̅|C) and shows that: that is, the function S(x) is self-reciprocal:

Using this reciprocal property of S to calculate inverse probabilities avoids the problem that there is no quotient operator in Boolean logic.

Dependencies can be logical or temporal. Causality (a temporal dependence) also implies the possibility of recursion: this is clear from the “Chicken & Egg” conundrum, on which see Simmons et al.[39] (who consider whether hummingbirds pollinate lots of species because there are lots of hummingbirds or whether there are lots of hummingbirds because there are lots of species to pollinate) or Harvey et al.[40] (who consider two forms of recursive cyclic network to show how subtle recursion relations may be). The chicken entails a prior egg, which entails a prior chicken …

Formally: recursion is present when p(A|B)>0 and p(B|A)>0 both together. Conversely, recursion is not present when C (see Eq.2b) is independent of either A or B. This is because expressions like p(A|C) imply that all dependencies are subsumed in C. That is, if A depends on B then there exists a C that will say this with no loss of generality.

Here we will informally derive the “Hyperbolic Sum Rule” that we will prove formally in §4.1.

There is an apparent dilemma (when applying Bayes’ Theorem) of having to make what appears to be an arbitrary decision as to which conditionality is the prior. Indeed, when considering the classic “Chicken and Egg” dilemma, having argued in one way, we undergo a Gestalt change and suddenly find ourselves equally arguing in the opposite manner! The recursiveness of this dilemma is self-evident; a different prior may be appropriate depending on where the analysis commences. Curiously, the case of the two-layer Fabry-Perot cavity (Eq.1c) involves optical events that lie on each other’s light cone; that is to say, as a physical phenomenon it is directly on the boundary of being causally connected or disconnected. In this case an intrinsic ambiguity of priority exists. If either may be prior then neither must be. That is, for a recursive dependency of {A,B}, p(A|B)=p(A) and by symmetry we also must have p(B|A)=p(B).

In any case, issues of the probability of events and their priority are frequently set within a spacetime context, with the associated physical implications. One of the aims of this paper is to establish a solution of this dilemma of choice conforming to Special Relativity but still recognisably consistent with conventional probability theory.

Note that even though both expressions p(A or B | C) and p(A and B | C) commute for A and B, Bayes’ Theorem does distinguish the precedence of the conditionalities associated with the primary events.

Eq.2b can be expressed in one of two ways, according to which of the primary events A and B is prior: substituting Eq.2c into Eq.2b we find two choices offering two possibilities for the logical OR function:

Eq.2d may be simplified by noting that the component term of Bayes’ Theorem appearing at the end of each CSR expression (each such latter term representing a conditional possibility) can also be assigned a probability associated with the truth of that conditionality. This allows us to informally re-write Eq.2d as a single statement, by expanding upon the products from Eq.2c: where the subscripts {A, B} indicate the assumed prior, and the expression in each of the pair of square brackets represents the respective Bayes’ Theorem choice. Eq.4a can be regularised by including the probability associated with the prior, that is, now explicitly including a recursive element. Note, the probability p(A|BC) associated with the prior is not the same as the probability of the prior, which we may state informally as: p(“A is prior to B”). Thus, we explicitly include the probabilities associated with the priors (removing the subscripts {A,B}). In effect we now assume the possibility of a ‘prior prior’ and the (logical/temporal) ordering of the events {A,B} now also starts to be explicitly taken into account: allowing a consistent (and symmetric) simplification which intrinsically includes the possibility of recursion:

In Eq.4c we have the probability of “A or B given C“ being {[the probability of A given C] or [the probability of B given C]} reduced by a cross-probability factor F, where: which could be thought to represent a “first level” of dependency, that is, the dependence of A on B (and vice versa). But what if we also consider a deeper dependence between these events? This would give: or, continuing to full recursion:

Given that probabilities may not be greater than unity this infinite series can be compactly written as:

However, Eq.6 may be simplified since, given that it cannot be determined which of {A,B} is prior, we may take p(A|B)=p(A) and p(B|A)=p(B). Hence:

Eq.7 (isomorphic to Eqs.1) states the “Hyperbolic Sum Rule” (HSR) theorem to be proved where {A,B} are recursively dependent. In deriving the form of Eq.7, we asserted that p(A|BC)= p(A|C) and p(B|AC)= p(B|C); that is, due to the recursion between events {A, B}, the probability of A becomes essentially independent of B, and vice-versa. That is, Eq.6 can be re-expressed in the strictly hyperbolic form of Eq.7. In effect, this assertion (p(A|B)= p(A), etc.) can be justified as the most likely (MaxEnt) relationships between the conditional probabilities for A and B when we know a priori that A and B have a recursive dependency, but we don’t know the precise (quantitative) relationship between A and B. In Appendix C we prove the MaxEnt character of Eq.7. That is to say, these assertions can also be justified by recourse to the Principle of Indifference (PI) and the simplest non-informative prior in Bayesian probability (see [refs. 33, 35]).

Eq.5b expresses the case where the relation between events {A, B} is not recursive but stops at a countable depth or iteration: the overt conditionality between them is therefore maintained such that p(A|B)≠p(A) and p(B|A)≠p(B). For example, if one wanted to formalise the psychological aspects of Poker one might express a “bluff” as the F term in Eq.5a, a “double-bluff” as the F² term, and a “triple-bluff” as the F³ term etc. But the HSR of Eq.7 expresses the recursion needed for the etalon of Eq.1c (for example). In contrast, it is clear that the CSR of Eq.2b is the truncated HSR case of Eq.6, where the CSR a priori ignores recursion and its associated higher-order dependencies (either due to specified information, or because their existence is simply not taken into account) and excludes the higher-order (logical/temporal) ordering of propositions. In any case, Bayes’ Theorem (the Product Rule) remains valid.

Both the HSR and the CSR are MaxEnt in each of their respective domains of applicability (proved respectively in Appendices C and E), and we need to know when to use the CSR and when the HSR (and why). In particular, the point of this paper is to emphasise the importance of the temporal (or logical) ordering of propositions, and therefore the possibility for recursion, which is what distinguishes between the CSR and HSR: the CSR doesn’t adequately take the temporal ordering of propositions into account, whereas the HSR does. And, this is particularly evident in the Venn diagram of Figure 1 which cannot represent the temporal or logical ordering of the propositions.

As an example, we consider the scenario of a fair coin being tossed twice. We designate the event A as “the first toss gives a head”, and the event B as “the second toss gives a head”, and we wish to compute the probability of the first toss or the second toss giving a head. Such a ‘fair’ coin does not have memory (each toss is independent of any others), so that we have p(A) = p(B) = ½. Here there is no recursion since p(C|A) = p(C) and p(C|B) = p(C): that is, C is independent of both of {A,B}, and therefore we may use the simple form of the CSR (Eq.2a): p(A$\mathrm{o}\mathrm{r}$ B) = ½ + ½ – ¼ = ¾.

Clearly, applying the HSR (Eq.7) to get p(A$\mathrm{o}\mathrm{r}$ B|C) = 4/5 in this context is manifestly false, precisely because we already know that there is no recursion present. The difference between the hyperbolic (HSR, Eq.7) and conventional (CSR, Eq.2b) sum rules is specifically the prior information that must be taken into account to do the calculation. The decisive question is whether or not the mutual dependence of {A,B} is recursive.

There are other simple cases where memory is involved, such that p(A|B) ≠ p(A). One such case is Cox’s [ref.7] example of randomly drawing black or white balls from a bag. The probability of a result is affected by the results of previous draws, but this is a memory effect, not a recursive one; and in this context, where the system only iterates based on the forward propagation of a previous state (i.e. no feedback), see Appendix F for a discussion of FIR (finite impulse response) filters. In general, if p(A|B) and p(B|A) are expressed in non-recursive form the CSR should be used, but if not then the HSR should be used. The CSR should be used only where recursion can be a priori excluded.

Consider the function Σ:

$\mathsf{\Sigma }\left[\mathsf{\Sigma }\left(x\right)\right]=x$, that is: Σ is self-reciprocal, or $\mathsf{\Sigma }\left(x\right)={\mathsf{\Sigma }}^{-1}\left(x\right)$. It is also doubly differentiable and a continuous, monotonically decreasing function: $\partial \mathsf{\Sigma }/\partial x=-2/{\left(1+x\right)}^2$. The inspiration for Eq.8a comes from Cox’s original equation for S as another self-reciprocal function (see Eqs.3 above) which is also doubly differentiable [7]. In Cox’s case, he also considered the situation where S (and x) are raised by an integer power m, such that S^m = 1 − x^m for integer m. For even m>0 this can be factored into the geometric series: with the series in parentheses representing an infinite summation (as m→∞) of the quantity (1+x)⁻¹. Note that this treatment is quite general, such that x may be complex. However, for all m (with integer m≥1), we can also write:

Recognizing that for even m the roots of S^m lie symmetrically around the four quadrants of the unit circle centred on zero in the complex plane, allows us to additionally exploit the associated symmetry relation for the n^th root, x_n= −x_m/2+n. It is important to realise that results from complex analysis underpin this analysis (as we also mention below following Eq.10c, and see Parker & Jeynes 2023 [ref.37] for a fully complexified treatment).

Therefore, substituting x_n=−x_m/2+n for the m−1 terms of the series summation in Eq.8c, we can write:

Applying this (counterintuitive) symmetry we see that Eq.8a is then simply the limit of Eq.8d as m→∞. In this limit the distinction between even and odd m is insignificant. In any case, it is clear that equivalent to Cox’s first solution (using m=1) the quantity S=1−x, represents an important term in the factorisation of Σ, but is by no means the only available solution in the general case for the roots of the function Σ.

From de Morgan’s Theorem ($\overline{A \mathrm{o}\mathrm{r}B}\equiv \overline{A}\overline{B}$, hence $A \mathrm{o}\mathrm{r}B\equiv \overline{\overline{A}\overline{B}}$): where $\overline{B} ($the negation of B) is required in the conditioning probability of the first probability term in order to satisfy the appropriate application of the Product Rule in de Morgan’s Theorem.

Substituting Cox’s first-order expression for Σ (that is, Σ = 1 − x) we recover the “Conventional Sum Rule” (CSR, Eq.2b), after some manipulation. However, substituting the full bilinear form for Σ (Eq.8a) into Eq.9: and expanding the outer self-reciprocal Σ function:

In the recursive case (as we have previously discussed for Eq.7), the probability of A becomes, in effect, independent of B; $p\left(\left.A\right|\overline{B}C\right)=p\left(\left.A\right|C\right)$, such that Eq.10b can be simplified to: which is Eq.7 as required.

Note that although the intermediate Eqs.8b,c,d (and the implied complex analysis) are not used in the derivation of the HSR (Eq.7 or Eq.10c), analogous complex quantities make an appearance in the discussion of how the finite impulse response (FIR) filter offers a good exemplar of the CSR (sketched in Appendix F). The standard theory of such filters (which are important for digital signal processing applications) fundamentally relies on the theorems of complex analysis.

Having derived a “Hyperbolic Sum Rule” (HSR) expression, it is interesting to see how much it diverges from the “Conventional Sum Rule” (CSR) in a simplified scenario that considers the function p(A or B) for two possible and equally probable events {A, B}, that is, p(A)=p(B).

The HSR is well-behaved in that it yields the same results as the CSR in the limit that neither event occurs, that is, p(A)=0 and p(B)=0. In this case, we have p(A|BC)= p(B|AC)=0 (that is, p(A AND B|C)=0 from Bayes’ Theorem). It is straightforward to see in this specific case that p(A$\mathrm{o}\mathrm{r}$ B) = 0 for both sum rules.

The HSR is also well-behaved where p(A)= p(B)=1. In this case, the CSR reduces to p(A $\mathrm{o}\mathrm{r}$B)=p(A)+p(B)-p(B) =p(A)=p(B). The HSR also reduces to the same result, but via the progression p(A $\mathrm{o}\mathrm{r}$B|C)=2p(A|C)/2 =2p(B|C)/2=p(A|C)=p(B|C). For the case where we might want to include the result p(A|B)=p(B|A)=1, then we employ these in the denominator of Eq.6 (we must include explicit knowledge – see §2.3) to get the same (correct) result again.

For the simple system in which there are two equally probable events {A,B} the conventional and hyperbolic forms of the sum rule converge when the events are either impossible or certain. It is for the intermediate probabilities 0<p(A)<1 that the two forms diverge. Figure 2 shows the behaviour of p(A$\mathrm{o}\mathrm{r}$B); calculated for the simple case of p(A)=p(B) and also assuming p(A|B)=p(B|A)=p(A)=p(B) for HSR. For CSR the two events {A,B} are also assumed to be independent, that is, p(A|B)=p(A) and p(B|A)=p(B).

From a simplistic Maximum Entropy (MaxEnt) argument, one might expect that the greatest difference between the two sum rules would occur when the events have the most uncertainty, that is, p(A)=½. For this situation, the CSR returns an overall probability of p_CSR(A$\mathrm{o}\mathrm{r}$B|C)=½+½−(½×½)=¾ (75%). In contrast, the HSR returns a slightly higher overall probability of p_HSR(A$\mathrm{o}\mathrm{r}$B|C)=(½+½)/(1+[½×½])=⅘ (80%). The presence of recursion (mutual dependence) increases the probability that both events will occur.

Interestingly, however, Figure 2 shows that the maximum difference between the two sum rules is actually 5.09 percentage points, for p=45.35% in this simplified case. (The asymmetry reflects the slight asymmetry of the hyperbolic sum rule.) Thus we see that the difference between the conventional and hyperbolic sum rules may be quite minor: in most cases it would be hard to observe (less than 5 percentage points).

Being MaxEnt, an application of the HSR of Eq.7 would be to calculate the sum probability for two events where the relationship between the two events is not known, yet some correlation (including recursion) is reasonably anticipated. In contrast, the CSR (also MaxEnt) explicitly requires knowledge of the conditional probability relationship between the two events: this may be estimated or simply assumed not to exist (since independence is often a reasonable approximation). Note that the CSR explicitly excludes recursion, an assumption which should be justified. Thus, the HSR of Eq.7 obviates the need for any unknown conditional probability, and also avoids any unjustified assumptions relating to recursion.

For example, imagine a scenario of elections occurring in neighbouring countries “A” and “B”, with leading candidates “Alice” and “Bob” in each, respectively. Opinion polls consistently suggest that both candidates each has a good chance of being elected: 45% in each case. We pose the question (to which we would like a Bayesian answer, if possible): what is the probability that at least one of the two leading candidates will be elected?

In order to apply the conventional sum rule we require the conditional probability that Alice will be elected given Bob is elected, or alternatively the probability that Bob is elected given Alice is successful (given by Bayes’ Theorem, Eq.2c). In the CSR context (in the absence of any other information to the contrary) the best we can assume is these are essentially independent events such that:

The CSR can then be applied to offer the MaxEnt probability of either Alice or Bob being elected (given that the two events are mutually independent) as:

However, the HSR allows us to include the possibility of a mutual (recursive) relationship between these two events (without making any other assumptions which could skew the calculation), giving a MaxEnt (most likely) probability in the absence of any other information (or any implicit assumptions):

Thus, the HSR probability estimation of at least one of the candidates being successful is noticeably higher than that anticipated from the CSR. This is because there are a variety of ways the two elections could potentially influence each other, and the HSR accounts for the possible existence of these mutual interactions, while (in the absence of specific information as given by the appropriate conditional probabilities) the CSR ignores such influences.

It is perhaps surprising that such a difference between the two sum rules has not been noted up to now: surely empirical observation would have shown a discrepancy from the (wrong) current theoretical calculation based purely on the CSR? That no-one has suspected that the CSR is faulty may be due to the fact that for most simple cases (e.g. card games, gambling etc.) recursion is generally absent (and the CSR is correct). Of course, probability theory originated in the analysis of games of chance where recursion is excluded. For more complex cases there is usually no possibility of re-running the observations (elections in this case) sufficiently to gain a statistically significant estimate of the associated empirical probabilities. Most real cases have complex (and recursive) conditionalities and dependencies, and most are unrepeatable, meaning that the associated probabilities cannot be determined by conventional methods. Thus, any systematic error in the underlying probability theory is probably unobservable, and it is hardly surprising that it has been unobserved and unsuspected until now.

In Appendix D we prove the following relations:

A version of Bayes Theorem for multiple summed hypotheses {X}_OR as the number of these hypotheses grows large:

We also exploit the hyperbolic tangent function to prove a Hyperbolic Sum Rule for multiple hypotheses with multiple conditionalities:

Finally, a generalised Hyperbolic Sum Rule with N multiple hypotheses, which sorts odd-ordered conjunctions into the numerator and even-ordered conjunctions into the denominator:

This may also be written in a form such that the probability distributions are represented (as MaxEnt decaying exponentials probabilities, see Jaynes 1982 [ref.14]) by the appropriate Lagrange multiplier β (noting that different distributions are generated by different choices of β, with an example of this given in a recent treatment of the Wine/Water paradox [ref.35]): additionally showing that the Hyperbolic Sum Rule is well-behaved for multiple hypotheses as $N\to \infty$.

In Appendix E we prove the following relation for the Conventional Sum Rule for multiple hypotheses:

In simple cases, the distinction between the Conventional Sum Rule (CSR) and the Hyperbolic Sum Rule (HSR) is probably unobservable, given the attainable empirical precision. The case of the two parallel elections considered in §3.2 has 80% for the HSR compared to only 75% for the CSR, when the two election probabilities involved are both 50% (p(A)=p(B)=0.5). But the CSR cannot model recursion.

However, the derivation of the HSR uncovers a relation that is also observed in diverse physical settings. In particular, the HSR is isomorphic not only to the hyperbolic tangent double-angle identity (Eq.1a) but also to Einstein’s famous velocity addition formula (Eq.1b) and to the reflectivity expression for an optical etalon (Eq.1c). Thus, the general (hyperbolic) probability sum rule conforms to a ‘template’ seen in the physical world; that is to say, although being apparently a logical relation it behaves just as many physical phenomena do. Therefore we can say that the etalon function (Eq.1c: intrinsically probabilistic in nature, being a wave scattering phenomenon) represents at least a physical embodiment of the HSR. In particular, the physical structure of the etalon with its two scattering interfaces (A and B, say) suggests that the reflection coefficient calculated from the solution to Eq.1c can also be interpreted as representing the physico-logical answer to the question: what is the probability that a quantum-mechanical particle (that is, a photon) is reflected from interfaces “A OR B”? It is clear that the basic two-faceted etalon physically expresses the logical OR operation.

This raises the interesting philosophical question: is the hyperbolic nature of the HSR a result of the intrinsic hyperbolic character of the natural universe? Affirming that it is implies that probability theory is firmly empirical (and not merely mathematical): both the theory of Special Relativity (Eq.1b) and the quantum-mechanical scattering probability (Eq.1c) are consistent with the hyperbolic nature of spacetime. But denying it would imply that it is simply coincidental that the HSR shares fundamental characteristics of the natural universe.

This question actually goes to the root of a deep philosophical debate as to the intrinsic nature of probability theory: is it purely an empirical construct where probabilistic variables only acquire meaning when a statistical measurement takes place? This view is implied by the so called ‘frequentist’ fraternity, who do not ascribe an independent reality to the “laws” of probability; rather they consider that the probability of a hypothesis depends solely upon the frequency of success over a given number of trials. But Bayesians would say that the hyperbolic nature of the HSR implies that probability theory and the underlying theoretical equations form a component part of the full set of universal natural laws; that is to say, they are an intrinsic aspect of the (hyperbolic) universe with their own independent existence.

We should add that the result that the MaxEnt nature of both the HSR of Eq.7 (recursive, see Appendix C) and the CSR (non-recursive, see Appendix E) indicates that the equilibrium state of such physical systems (corresponding to a stable and most-likely configuration) obeys both the kinematical Principle of Least Action as well as the entropic Principle of Least Exertion (ref.[10]). These fundamental principles underly all known physical phenomena and therefore indicate that both the HSR and CSR are consistent with the requirements of the physical world. The fact that the HSR is MaxEnt also implies that all the parts of such systems recursively condition all the other parts. That is, in the general case everything is entangled with everything else just as Karen Barad implies [ref.31]. But here we wish to avoid metaphysical discussion (for which see [ref.6]).

Why has the issue of recursion apparently not previously been explicitly considered in probability theory? This may simply be an accidental result of history, with probability theory having developed from the gaming scenarios associated with cards and dice, where events are discretely independent and are characterised with shallow conditionalities that do not reach far back into the past. For example, the game of Poker has explicit calculable conditionalities based on cards revealed to the table, cards in your hand, and the hidden cards remaining; the calculable conditionalities do not involve the (fascinating) issues of bluffing, double bluffing, and multiple bluffing. These could in principle be expressed logically (using the formalism sketched above), but are currently treated as psychology.

However, it is interesting to note that Cox (1946 [7]) derives the general result S^m=1−x^m for the self-reciprocal function S(x), stating that the value for m (assumed an integer) is “arbitrary” and “purely conventional”; indeed, Jaynes writes that the value of m is “actually irrelevant” [ref.1]. For “simplicity of notation” Cox arbitrarily chooses m = 1, which is both convenient and intuitive, since it immediately leads to the Conventional Sum Rule associated with the Venn diagram of Figure 1. Thus Cox’s analysis allows for recursion conceptually, although he apparently appreciated neither this nor the physical interpretation of his function S^m.

But exploiting the hyperbolic insight described here, we can now understand m to represent the degree of mutual dependency leading to recursion; that is, it has both a logical and physical (spatio-temporal) meaning, with real physical implications which are not at all “arbitrary”. In particular, m=0 is a trivial case, m=1 is the simple CSR case, m^-1=0 is the HSR case. We consider this further in §4.3.

Another reason for ignoring recursion is that the appropriate calculus of probabilities requires substantial computing power which has only become available in recent decades. Up to the point of the emergence of artificial intelligence (AI) and machine learning (ML), approximately at the turn of the millennium, there hasn’t been a good reason to develop a theory of recursive probabilities, since before this time the necessary computations were practically intractable. However, as large-scale computing power has evolved over recent decades, this has also enabled the development of the sophisticated (Bayesian) algorithms underpinning AI and ML programming. Such algorithms allow a dispassionate approach to making probabilistic inferences and decision-making in an environment of uncertainty, within a mathematical framework for updating beliefs and reasoning about uncertain events. In addition, recursion provides AI with additional flexibility to handle complex and dynamic problems by recursively decomposing them, while leveraging self-referential architectures and utilizing iterative processes.

Recursion can be exploited relatively simply in AI as a means to break down complex problems into smaller, more manageable subproblems. Alternatively, the complexity and power of modern computing also enables the employment of recursive data structures that can be used to represent hierarchical or interconnected relationships. Recursion can also be seen in the inductive reasoning aspect of AI, which involves inferring general rules or patterns from specific observations, so as to learn iteratively from data in order to improve the accuracy of predictions or classifications.

In AI (and the associated field of neural networking) recursion has been employed empirically: the complexity associated with the algorithms and feed-back routines aren’t generally amenable to analytical (closed-form or tractable mathematical) analysis, such that currently only ‘simulations’ can be used to analyse AI behaviour. It is in this context of recursive AI that the Hyperbolic Sum Rule described here allows the systematic, reliable and consistent handling of multiple recursive hypotheses (or events). We therefore expect the HSR to enable a more analytic (closed-form and tractable) analysis of complex AI algorithms to give additional insights into their form of operation and anticipated outputs.

We have shown that in different situations different rules apply for summing the probabilities of events (or, nearly equivalently, the probabilities of certain hypotheses being true). We wish to find a relation that allows us to compute p(A or B|C), that is, the probability of (A or B) given C, where {A,B} are events (or hypotheses) and {C} are conditionalities.

There are two distinct cases depending on whether or not {A,B} are recursively dependent. If {A,B} are independent (such as coin flipping, see §2.4) then the “Conventional Sum Rule” (CSR) applies. But if {A,B} are recursively dependent (such as reflection from a multilayer stack, see Eq.1c) then the “Hyperbolic Sum Rule” (HSR) applies. Concatenation formulae for either or both of multiple events {A,B, …} or multiple conditionalities {C,D, …} are readily derived for both the HSR (Appendix D), and the CSR (Appendix E; also see the summary in §3.3 of important results for both HSR and CSR). Therefore there is no loss of generality from treating only the simple cases.

If p(A|B)>0 and p(B|A)>0, with both probabilities being valid independent of any temporal (that is, causal) or logical constraints, then {A,B} are recursively dependent and the HSR applies. Otherwise the CSR applies. Conversely, the CSR applies if the probability relation {p(C|A)=p(C) or p(C|B)=p(C)} holds.

Note that in simple cases (see Figure 2) the difference between CSR and HSR is probably too small to be easily observed: even in the fair coin or election cases (§2.4, §3.2) the CSR gives 75% where the HSR gives 80%.

It is clear that the CSR and HSR are formally related (see Eqs.8). It also seems fairly clear that the “truncated expansion” of Eq.8d has properties related to (for example) the “finite impulse response” (“FIR”) filter that is important in digital signal processing (DSP) applications (further details in Appendix F). The case represented by finite m where m>1 and m^-1>0 (analogous to the FIR filter) offers an engineering approach to the probabilistic treatment and control of system noise within a finite (and specified) timeframe - an issue of practical importance for stochastic systems. Such practical probabilistic cases, formally CSR (see Appendix E), blur the distinction between CSR and HSR, since it seems that such “FIR filtering” may be represented approximately as a “truncated HSR”, with a formalism that is also approximately MaxEnt in its own terms (see Appendix C). However, the derivation of the HSR is directly from Eq.8a: the discussion of finite values of m in Eqs.8b,c,d is presented only by way of comment.

The following supporting information can be downloaded at the website of this paper posted on Preprints.org.

The Hyperbolic Sum Rule (HSR) for probabilities is intended to express the most-likely (Maximum Entropy) sum of probabilities where the conditional probabilities associated with the conventional sum rule are not available and where the recursive properties are not explicitly known. That is to say, in the absence of the additional information (associated with the conditional probabilities as well as recursion) required to define the conventional sum rule, we can still define the most likely addition of the probabilities using Maximum Entropy.

The following function (which is equivalent to the Partition Function in statistical mechanics) is known to be MaxEnt (see Appendix I of Jaynes 1982 [ref.14]). We first discuss here why it is MaxEnt. The function f(z₀, z₁, z₂ … z_i … z_N) consists of a series of N+1 terms, whose summation is given by the quantity Z:

or

The free parameter λ is the essential controlling variable of the function f, expressing the ‘shape’ (or rate of exponential decay) of the function, while the index i is constrained to be an integer, with the maximum value N representing another free parameter of the function. The entropy H of the function f is given by term-by-term summation of the Shannon entropy of each ith component:

Each ith component z_i is associated with an ith state of the system f; that is to say, the function f can also be thought of as describing the relative ‘probability’ (or propensity for existence) of each of the components comprising a system of N different (distinguishable) states. In which case, the ith component z_i can also be considered to represent a measure of the ‘relative strength’ of the ith component of the system. The fact that the system may be characterised with a finite entropy H means that we can assume that the number of states of the system is finite and has a granularity (that is, it is discretised, or quantised).

Consequently, this also means that there exists a quantised number E, representative of the overall system, where the discretisation (granularity) is represented by the quantum ΔE, such that the overall total number of quanta Q existing within the system is $Q=E/\mathsf{\Delta }E$. By inspection, it is also clear that $Q\equiv Z$. How these Q quanta are distributed across the function f represents a constraint to the system as we maximise its entropy. We therefore associate a quantity E_i with each of the component terms of f such that the product of z_i and E_i for each component represents a measure of the strength of the ith state, and by summing across all the N+1 states we obtain the overall quantity E:

which can be re-written as:

Following the standard treatment of Caticha[41] we employ Lagrange multipliers α and λ, such that we set the independent equations to zero, we can use Eqs.A1b, A2 & A3b to consider the variation of the system entropy H with z_i so as to find its maximum:

We can swap the order of the summation and differentiation operations to evaluate the derivative of each of the i terms in Eq.A4:

To maximise the system entropy H the ith component z_i of the function f must be simply given by:

That is to say, a distribution f is Maximum Entropy if its constituent terms follow a decaying exponential profile (see Eq.A1a).

The most basic sum rule function S₀(A,B)=A+B is Maximum Entropy (MaxEnt), since it is easy to demonstrate that the summation of any two terms can be made to follow the decaying exponential template of Eq.A6. For example:

which is simply the two terms of a N=1 series, for i=0 and i=1. That is to say, we can express the ratio B/A as being equivalent to exp(−λ) for an appropriate value of λ, while A is equivalent to Z/exp(1+α) for an appropriate value of α. Of course, the converse is also true:

where for consistency, we employ the same value for λ as in Eq.B1a, but the values for α and Z are as appropriate. The simplest sum rule of Eqs.B1 is therefore easily found to be MaxEnt: are there any other sum rules that are also MaxEnt?

Clearly, any such sum rule has to obey the template of Eq.A6 in order to be MaxEnt. The next obvious sum rule that could obey Eq.A6, and which only employs the two parameters A and B is given by the following geometric progression of N+1 terms:

There are two aspects to this sum rule that need to be noted: it only converges for B<A as N→∞, and it is clear that the overall summation can easily sum to greater than unity for various values of A and B. In particular, this latter aspect can be seen by re-writing Eq.B2a as:

That is to say, the first two terms of the series come from the most basic sum rule, and there then follow potentially an infinity of additional terms, such that the overall summation could well be greater than unity. Thus as a general probabilistic sum rule, it would appear be inadmissible. Of course, the case where A<B, requires the converse sum rule:

These geometric progressions offer a closed form analytic solution:

and

Clearly, both forms for the sum rule have a pole (singularity) when A=B, such that the overall summation will be greater than unity. These are indeed ‘acceptable’ MaxEnt solutions as per Eq.A6, but are clearly deficient as probabilistic sum rules, since the two rules each need to be judiciously selected (i.e., disjoint application of the two rules is required according to which of A and B is the greater), and they can also sum to greater than one. As such, we can exclude the sum rules S₁ and S₂ as an appropriate general MaxEnt probabilistic sum rule. Indeed, the initial sum rule S₀(A,B) can also be excluded as an appropriate probabilistic sum rule for this same reason, since it will also frequently sum to greater than unity.

It is interesting to note that the conventional probabilistic sum rule (employing conditional probabilities) always lies within the range of zero and one:

That is to say, the first two terms of the conventional sum rule (CSR) are equivalent to the basic sum rule function of Eqs.B1, but there is an additional third term to the rule, which ensures that the overall quantity behaves ‘correctly’ as a probabilistic quantity, such that it never exceeds unity. However, this is at the cost of introducing an additional term (indeed, additional information), alongside A and B: the conditional probabilities B_|A or A_|B (depending on which probability is considered prior). Such a conditional probability is essentially an additional (third) parameter to the sum rule, such that the function in effect becomes a sum rule requiring three input arguments, rather than just the two parameters (A and B) that we have been discussing above. From this perspective, the conventional probabilistic sum rule of Eq.B4 cannot be MaxEnt (apart from the fact that it doesn’t follow the template of Eq.A6), because it requires additional information (based on the conditional probabilities); whereas the rationale for a Maximum Entropy sum rule is to sum (in the most likely manner) the two parameters A and B in such a way that the minimal assumptions as to how A and B might depend upon or be related to each other is made; that is say, the MaxEnt sum rule requires the least additional information over and above the values for A and B so as to provide their most likely sum value in the absence of any other information.

In addition, it is worth noting that when the events A and B are mutually exclusive, then the conventional probabilistic sum rule of Eq.B4 degenerates to the basic sum rule of Eqs.B1. However, although the basic sum rule is MaxEnt (as we have previously discussed, Eqs.B1) the underlying reason for the degeneracy in this case of Eq.B4 means that we now know the exact (mutually exclusive) relationship between A and B. In general, the conventional sum rule of Eq.B4 is not MaxEnt (since it requires additional knowledge or a best-guess estimate of the (conditional) relationship between A and B); but the knowledge that A and B are mutually exclusive causes the conventional sum rule to degenerate to the basic sum rule (with its MaxEnt properties), but which is still inadmissible as a general probabilistic sum rule since it frequently leads to probabilities greater than one. However, Appendix E shows that in the limit where the conditional probabilities are assumed identical to the independent probabilities, then the CSR also becomes MaxEnt; however, this is a special case of the CSR where Bayes’ Theorem (as a form of the Product Rule) becomes trivial.

However, there still exists another potential probabilistic sum rule, which offers still better behaviour. This function emerges as the generalised expression arising from Cox’s treatment. Fortunately, the MaxEnt aspect now becomes apparent as we can show here that the Hyperbolic Sum Rule (HSR) is a well-behaved maximum entropy function. We start by expressing it in its basic form:

We first notice that the hyperbolic sum rule is isomorphic to the hyperbolic tangent double-angle identity:

Just for clarity, we remind ourselves that we use A and B here as shorthand for p(A) and p(B); that is to say, A and B each represent an event, and p(A) and p(B) each represent the respective probabilities for events A and B to occur. Thus, when we employ the symbols A and B in Eq.C1a, this is actually a shorthand for the probability functions p(A) and p(B) associated with each of the events A and B. Thus, when in Eq.C1b we explicitly employ the hyperbolic tangent (tanh) functions of a and b, we can immediately see the isomorphism between p(A) and tanh(a), as well as p(B) and tanh(b). This means that we can therefore also assume that S_HSR(A,B) is isomorphic to the function tanh(a+b). This is a longwinded way of saying that the probabilistic hyperbolic sum rule S_HSR we are seeking to analyse must be identical to the hyperbolic tangent (tanh) function.

Using $\tanh \theta =\sinh \theta /\cosh \theta$, as well as $\sinh \theta =½\left\{\exp \theta -\exp \left(-\theta \right)\right\}$ and $\cosh \theta =½\left\{\exp \theta +\exp \left(-\theta \right)\right\}$ then it is straightforward to re-write Eq.C1b as: where in this case $\lambda =2\left(a+b\right)$. We can immediately see that the hyperbolic tangent sum rule identity is composed of the quotient of two functions, each of which conforms to the template of Eq.A6, so that each of the two functions composing the quotient is MaxEnt. However, this immediately poses the question, is the resulting (quotient) function therefore also MaxEnt? We can binomially expand the denominator function of Eq.C2 as an infinite series as follows:

Multiplying out the brackets and grouping the terms:

Eq.C4 closely resembles the template MaxEnt function of Eq.A6, except for its initial term, which is ‘wrong’ by the constant amount of one. That is to say, we can re-express Eq.C4 as:

It’s interesting to note that since the discrepancy is a constant (−1 in the case of Eq.C5) then all the results emerging by employing $Z\equiv \tanh \left(a+b\right)$ as a moment generating function (that is, to calculate quantities such as the mean and variance) are the same as if Z really were a MaxEnt function according to Eq.A6. This can be alternatively understood by approximating Eq.C5 to a finite series of N+1 terms, and dividing the constant overall ‘discrepancy’ of −1 amongst all the N+1 terms as follows:

We then take the limit as N → ∞, such that Z → tanh(a + b), where the limiting value for each term z_m of the infinite series is:

This makes it clear that in the limit as N → ∞ the limiting value for each term z_m does conform to the MaxEnt template of Eq.A6, hence tanh(a + b) is MaxEnt and the isomorphic HSR S_HSR(A,B) is also MaxEnt.

Thus we have proved that the Hyperbolic Sum Rule represents the MaxEnt (most likely) value of the probabilistic addition of two probabilities (i.e. the probabilistic OR function) in the absence of any additional information regarding the relationship between the two events A and B represented by the two probabilities to be summed. This HSR S_HSR is well-behaved, since it always unconditionally yields results between zero and one for the sum of any conventional probability quantities (lying between zero and one).