Preprint
Article

Kullback’s Minimization of Information Discrimination as the Action Principle for Biological Systems

Altmetrics

Downloads

112

Views

29

Comments

0

This version is not peer-reviewed

Submitted:

08 May 2024

Posted:

09 May 2024

You are already at the latest version

Alerts
Abstract
The identification of an action principle for directing basic system energy exchanges can be used in a general framework for the study of dynamics in systems. The application of such action principles using Lagrangian procedures has become a very powerful methodology for the analysis of complicated problems in modern physics. This form of analytics has recently been considered as the mapping of the system state information in the context of system constraints. The process of translating perceived information into adaptive responses is considered to be the driving action dynamic for the living system. Reconciliation processes that provide for a minimization of the divergence of this information signature from a state of system stability are preferred. Such a state is naturally that of maximal entropy in the context of the operational constraints of the open living system. A mathematical expression incorporating inferential information entropy processing into biologic replicator dynamics describes the mechanics of the perception-action adaptive processes of living systems. From this framework, Kullback’s Principle of Minimum Information Discrimination is identified as the innate axiomatic action principle for guiding the dynamics of biological systems.
Keywords: 
Subject: Biology and Life Sciences  -   Biophysics

1. Introduction

In the formulation of Newtonian mechanics, “force” drives all dynamics according to prescribed physical laws. In the 19th Century, “energy” as an integral of the potential and kinetic capacity for change emerged in expressing phenomenal dynamics in the context of system constraints. In modern physics, the quantity termed “action” delineates the specific way the kinetic and potential energies of a system are transacted during any dynamic changes [1]. As such, phenomenal dynamics or actions are often described using generalized variational or extremum principles as opposed to specific physical laws. Variational principles use the calculus of variations to find functions that optimize variable values. It is considered as an alternative method for defining the dynamics of a system by bounding its functioning with the smallest free energy variations [2].
Therefore, these fundamental principles provide the methodological procedure for inferring the optimal probability distribution of system parameter values in a way that most constrains the energy variations during system changes. In this way, the processes of the system are directed toward the extreme elimination of any nonequilibirum conditions within the context of the system’s constraints and structural limitations [3]. The goal for using this approach has been to formally integrate these principles into the fundamental canonical structure of physical theory [4]. This analytic approach also allows for the mathematical examination of highly complex systems when there is limited information about the organizational and driving mechanisms. However, a general action principle for the dynamics of biological systems has not been elucidated. This paper proposes such a fundamental principle for biology and provides a rationale and framework for its functioning based on established physical and logical tenets.

2. Materials and Methods

2.1. The Historical Development of the Principle of Least Action

The principle of least action as first formulated by Pierre Louis Maupertuis, Leonhard Euler, Joseph Louis Lagrange and later refined by William Hamilton using Lagrangian mechanics, minimizes energy excursions during dynamic transitions and has historically been considered to be the primary variational principle for most natural physical continuum systems [1]. This approach informed the structure of Einstein’s Theory of General Relativity and Richard Feynman used an analogous logic in his “sum over all possibilities” or path integral formulation of quantum theory [4]. The rationale for employing a least action principle in physics comes from considering the path of the dynamics at every point in space and time. The mechanics for the procedure is based in the microphysical motion of system constituents in a way that that uniquely minimizes the total action of the dynamics. The trajectory path is determined by the force provided to the object at each point with the setting of an explicit resultant velocity in a specific direction. Since the measure of velocity is comprised of distance and time then any deviation from that direction (an alternative path) would add to the distance, change the velocity, and violate the conservation of energy. Because the path taken is a logical consequence of the general path minimization procedure instantiated in the calculus of variations, the principle of least action can be viewed as a basic axiom that uniquely connects physical theory to the mathematical apparatus used in the analysis of such systems. Thus, the principle of least action can be inferred from basic logical reasoning rather than arising from some new fundamental or teleological tenet. Rather than depending on unseen forces for action-at-a-distance as codified in the Newtonian laws, Lagrange, Hamilton, Maxwell and others have rederived phenomena dynamics by the identifying action functionals driven by energy balances as a more general framework [5].

2.2. The Entropy Principle as an Action Principle

Similar to the principle of least action, the entropy principle specified within the 2nd Law of Thermodynamics constrains the way that energy can change within a system under a given process or condition [6]. Shannon information is also considered to be closely aligned with the physical notions of entropy as a measure of the global system uncertainty [7]. It has been suggested, that when considering the principle of least action from a probabilistic perspective there is the potential course for a more fundamental derivation of the 2nd Law of Thermodynamics in terms of information theory [6]. For example, the dynamics of entropy can be described as a summation ensemble (integration) of all possible motion trajectories of a particle that is be represented by the general Shannon entropy information path H.
H a , b = k = 1 p ab k ln p a b ( k )
The most probable trajectory is the one yielding the maximum entropy distribution and is also considered the path of least action. Such an entropy trajectory assessment suggests that the 2nd Law of Thermodynamics is actually a statement of the statistics of entropic dynamics. The most conservative probability distribution of parameter values can be inferred utilizing the Principle of Maximal Entropy as described by E.T. Jaynes [8,9]. In this inference procedure, the probability distribution is determined by the maximum entropy permitted with the available information and known constraints. This type of information inference method has been used in the successful derivation of quantum mechanics, general relativity, and a variety of other physical phenomena [10,11,12,13,14]. Most significantly, Jaynes derived a proof for the 2nd Law of Thermodynamics that elaborated a transition of the microscopically based Boltzman formula to the macroscopic state using Liouville’s theorem [8,9]. Louisville’s theorem states that if all finite values of an analytic function are bounded, then the entire function is a constant function. The realized path of trajectory that these elements take is the integral sum of probabilities of all possible paths as a singular direction with minimal length and action in the context of system constraints. This Jaynes proof concretely connected the common notions of physical entropy with Shannon information theory.
Because of the significant implications of this finding, a rigorous axiomatic examination of Jaynes’ Principle of Maximum Entropy was performed by J. E. Shore and R. W. Johnson [15,16,17]. They determined that this principle is a logically consistent method of inference to conclusions when they are derived from new incoming information. Furthermore, they concluded that deductions made from other alternative measures of information content could potentially be inconsistent and lead to serious contradictions. Their analysis was not founded on just intuitive arguments or the considered properties of entropy as an information measure, but rather depended on more fundamental axioms containing the necessary properties for the inference procedure. These axioms were grounded in essential self-evident principles that required a consistent probability model of inductive inference and dependable results regardless of the solution course. The general axioms of Shore and Johnson included:
  • Uniqueness: The results provided by the inference should be unique.
  • Invariance: The choice of coordinate system is inconsequential.
  • System Independence: It is irrelevant whether independent information about independent systems is accounted for separately as different densities or together as a joint density.
  • Subset Independence: It is irrelevant if independent subset of system states is considered as a separate conditional density or as the full system density.
Based on these essential axioms, the Shore and Johnson proof demonstrated that under all conditions there is only one probability distribution solution that can be concluded with the introduction of new information into a system. Furthermore, that solution can only be attained by imposing a requirement for the maximization of system entropy as specified in the Jaynes principle [8,9,15,16,17].
The remarkable success of the inference methodology in examining the dynamics of systems has led Ariel Caticha and others to postulate that the laws of the physics are not laws of Nature but rather are rules for processing information about Nature [11,13]. This approach to describing dynamics also applies to chemical and biological systems [18]. In considering the future direction of scientific inquiry, Nobel physicist Frank Wilczek suggests that “fundamental action principles, and thus the laws of physics, will be reinterpreted as statements about information and its transformations” [19]. From this perspective, information processing then becomes the real force that is driving all known phenomenal dynamics.

3. Results

3.1. An Action Principle for Biology

The difficulty in understanding the natural phenomenal dynamics of living systems comes from our limitations in applying the traditional reductionist approach in analyzing complex systems while still requiring that they are subjugated to basic physical laws. Employing a unique action principle that arises from the logical inference of basic information processing procedures that occur with biological dynamics may be the way forward for analyzing these complex open systems. In the latter part of the 20th Century, physical chemists Ilya Prigogine and Lars Onsager were awarded the Nobel Prize for their analyses of the nonequilibrium thermodynamics of open systems [20,21]. Joseph Kestin formalized these ideas for open attractor systems as the Law of Stable Equilibrium or the Unified Principle of Thermodynamics [22,23]. This unified principle states that attractor systems that are stable in the Lyapunov sense will naturally oppose applied gradients and move back toward an equilibrated steady state condition. For living systems acting as open attractors, this adaptive recoil toward the steady state is triggered by divergent information within the purview of the organism’s perception.
An alternative derivative of the Jaynes methodology discussed above that considers such divergent information within a system was formulated by Solomon Kullback as the Principle of Minimum Discrimination Information [24]. This principle delineates the procedural dynamics required in the reconciliation of informational differences as the system state moves towards an equilibrated steady state condition. As demonstrated by Jaynes, this inference driven impetus for change is equivalent to the entropic forces of the 2nd Law of Thermodynamics [8]. The Kullback information minimization derivation utilizes the concept of relative information as described in the Kullback-Leibler ( D K L ) formulation of system information divergence:
D K L = i = 0 n P i ( S 0 i S i )
D K L is considered the average of the "surprisal-difference" between the original condition with Shannon information entropy probabilities S 0 i and the new status. Therefore, it measures the information differences between a new system state and the prior reference state of the system. In the terminology of Shannon Information, where p is the current probability distribution and q is the distribution that is the entropy-driven end target of the change dynamic as directed by the Jaynes principle, the equation becomes:
D K L ( q p = i n l n q i p i q i = i n ( l n q i l n p i ) q i
By way of this solution principle, incoming information is used to infer the formation of a new parameter probability distribution that minimizes the discrimination from the original distribution as much as possiblef 0 {\displaystyle f_{0}}. Because phenomenal dynamics are based on relational interactions, physicist Carlo Rovelli considers the relative nature of the D K L   metric as the best physical equivalent of the Shannon information entropy measurement [25]. Through the incorporation of the Boltzman constant in the calculation, physical thermodynamic entropy measures are a basic physical property of the system. Statistical determinations of physical thermodynamic entropy then becomes a special use case for inferential information processing. By contrast, information can be formulated as a strictly relational dimension connecting the elements and processes of any system. Therefore, the information entropy is mainly epistemological while physical thermodynamic entropy can be considered as more ontological. Kullback’s Principle of Minimum Information Discrimination applies an axiomatic inference procedure to minimize all relative information entropy differences in a way that globally maximizes the system entropy as subject to the system constraints.
Living organisms continually observe new information within their biocontinuum for the purpose of system adaptation toward a state of stability and survival [26]. This process is essentially equivalent to the minimization of free energy in a complex system as it naturally moves to a steady state [27]. Therefore, by the Principle of Minimum Information Discrimination the new probability distribution of the system information entropy is inferred in a way which minimizes the discrimination from the original distribution as much as possible. The trajectory for this inference in living systems during the processing of biological adaptations can be projected and characterized by integrating Shannon information entropy into the operations of a standard replicator dynamic as first described by R.A. Fisher [28,29]. The derivation of this integration and conversion into a Kullback-Leibler information divergence format (   D K L ( q p ) has been previously described in detail [26,30] and is summarized as:
I q , p = D K L ( q p = i n l n q i p i q i = i n ( l n q i l n p i ) q i
I(p,q) is the information state.
q is the target state with a fixed probability distribution.
p is the time dependent probabilities of current state.
When q is a target goal state with a fixed probability distribution and only the probability of p is time dependent then:
d d t i n l n q i q i = 0
Therefore, differentiating D K L ( q p results in the following expression:
d D K L ( q p = d d t i n l n p i q i = i n p i ˙ p i q i
Where p i ˙ (rate of change of probability   p i ) is defined by the replicator equation as:
p i ˙ = ( f i P f P ) p i
Substituting this replicator expression into our derivative d D K L ( q p results in:
d d t I q , p = d D K L ( q p = i n ( f i P f P ) q i
Where fi(P) is the fitness of each type i in the population with fitness being a survival likelihood or probability characteristic in the context of the conditions of the environment.
Since the probability q i sums to one and f P is the same for p i a n d q i then:
d D K L ( q p = f P i n f i P q i = i n f i P ( p i q i )  
If we consider d D K L ( q p as the kinetic and D K L ( q p as the potential, then the ACTION functional (kinetic – potential) for all i elements is described as:
A C T I O N = i n f i P ( p i q i ) i n l n q i p i q i
Therefore, ACTION is defined as the integral summation of the Lagrangian integrand (the difference between the kinetic and potentials of information) and is naturally minimized by the Kullback’s Principle of Minimum Discrimination Information. This integral also serves as a functional to determine the trajectory of dynamics in the biological system.

4. Discussion: Foundational Origins of the Action Principle for Biology

As noted previously, the reconciliation of information divergences within systems as determined by Kullback’s Principle of Minimum Discrimination Information provides the natural driving force for dynamics within those systems. For open living systems experiencing a constant influx of materials and energy with resulting disrupting internal variability, this axiomatic driven reconciliation action principle determines the trajectory and evolution of the system dynamics. Therefore, the functioning of this living system action principle arises fundamentally from the dynamics of information inference processing as subject to the constraints and adaptive mechanisms of the organism’s objectives for stability and survival. Inference is the process of reaching a conclusion based on empirical evidences and a procedure for reasoning. Considering the proposed action principle, these aspects of inference can be drawn from two postulates of Kestin’s Law of Stable Equilibrium and Unified Principle of Thermodynamics as paraphrased below [22,31].
  • When a system undergoes an energy/matter distribution transformation with the removal of internal constraints, the final equilibrated state of the system will be independent of the order in which the constraints are removed.
  • An equilibrated system’s attractor forces will naturally counter incoming destabilizing gradients to move the system back toward baseline steady state conditions. Furthermore, any increase in intensity in this gradient force will result in a corresponding counter increase in the system’s opposition to movement away from that attractor baseline steady state.
More recently, the theory of the minimum rate of energy dissipation provides a similar perspective to that of Kestin [32]. This newer theory states that when a system is in its dynamic equilibrated steady state condition, the rate of internal energy dissipation is at a minimum in the context of its constraints. However, that rate is increased if the system is moved from that steady state as it moves to return its stable condition.
The empirical evidence required for inference of an end-state in living system dynamics comes from the observed global informational macrostate of the system at each point in the process of reconciliation of divergent information to conclusion as defined by the expression for D K L ( q p . This steady state endpoint is the system’s attractor target and is independent of which adaptive constraints are applied. Most importantly from an empirical consideration, this end-state is critically necessary for the organism’s survival and is consistent with our traditional understanding of the workings of thermodynamics in living systems. There is no organism to observe if an equilibrated steady state somewhere within the attractor basin is not achieved.
Kestin’s second postulate represents a shift in perspective regarding the causal driver of system dynamics and provides basis of the logic for inferential reasoning. As noted, for living systems a global coherent attractor induced steady state condition is imperative for survival. A dysequilibrium condition exists if anything drives the system away for this coherency. For these open systems, energy transfers resulting from new force gradients coming into the system are naturally countered by the requirements to move toward an equilibrated steady state dynamic as driven by attractor forces. These base attractor forces in the context of the system structure and constraints determine the maximum possible system entropy unless some overwhelming force that has moved it irreconcilably away from this state. The information dynamics guiding the reconciliation of state is considered the mechanism that drives a change to a conclusion during the inference process.
For organisms that autonomously determine their own attractor conditions, the life system organization driver becomes the action functional with an innate action principle for system coherency. The action functional procedure for inference reasoning requires a maximization of information entropy and a minimization of information divergence within the living system in the context of the adaptive mechanisms and constraints. In other words, it represents the most probable system arrangement with an inferred boundary condition in a way that generates the least information. Though the functional reduction of the living system’s information divergences is actively processed through the organism’s adaptation mechanics, the global axiomatic impetus for these dynamics is grounded in the entropic drive of the Kullback’s Principle of Minimum Discrimination Information acting as an action principle.

5. Conclusions

It has been proposed that information processing may be fundamental to any computational description of physical phenomena [33]. Lagrangian mechanics are frequently used to describe the dynamics of physical phenomena based on the least action principle as an informational computational directive. In this popular approach, the foundations of phenomenal dynamics can be more succinctly represented by procedures for information processing rather than depending on unseen forces and Newtonian laws. Information can then be considered the currency of action and inferential reasoning the driving force for system dynamics.
In this paper a biological action functional is derived that contains an ordering action principle uniquely suited for living systems. The entropic drive for a living system’s adaptive reconciliation of information divergence from its equilibrated steady state condition can be described by a mathematical formulation of biologic dynamics that incorporates the axiomatic inference procedure of Kullback’s Principle of Minimum Discrimination Information. This formulation is informed by an approach previously used in the analysis of population evolutionary dynamics in which the Kullback-Leibler information divergence metric ( D K L ) was integrated into a replicator dynamic to function as an inference engine [34,35]. Those investigators found the differentiating capability of the D K L and the mechanics of traditional replicator dynamics with a system fitness function creates a natural inference procedure with Bayesian learning for informing population distributions during evolutionary events. The current modified formulation is directed toward the action functioning of dynamics in individual organisms rather than populations. While the mechanics inherent in the adaptive processes of the derived equation naturally reduces information divergences in the system, the broader phenomenological and axiomatic rationale for these dynamics are grounded in the inference procedures and entropic drive of the Kullback Principle of Minimum Discrimination Information [30]. Therefore, this principle can be considered as the action principle for biological systems.
The complexity of biological systems has resulted in limited success in the application of traditional physical principles in the analysis of living systems. Emergent properties that are the result of inherent biological system nonlinearity and downward causation often result in seemingly purposeful activities that elude customary physical analyses. The delineation of a physical principle for action in living systems could lead to a new understanding of the mechanisms of biological phenomena. In this way, the ostensible teleonomic aspects of these systems are then more appropriately considered as operationally driven by programmatic objectives determined by global scripts as brought about by natural processes such as attractor conditions, entropy excursions, and information processing principles [26].

Conflicts of Interest

The author declares no conflicts of interest.

References

  1. Coopersmith J. 2017. The Lazy Universe: An Introduction to the Principle of Least Action. New York, NY: Oxford University Press. ISBN 978-0-19-874304-0.
  2. Niven RK, Bjarne Andresen B. Jaynes' Maximum Entropy Principle, Riemannian Metrics and Generalised Least Action Bound. 2011.
  3. Moroz A. The Common Extremalities in Biology and Physics: Maximum Energy Dissipation Prinicple in Chemistry, Biology, Physics and Evolution. 2012; 2nd ed. Elsevier, Inc. [CrossRef]
  4. Feynman R. The Principle of Least Action: The Feynman Lectures, 1963 https://www.feynmanlectures.caltech.edu/II_19.html.
  5. Penrose, Roger. 2004. The Road to Reality: A Complete Guide to the Laws of the Universe. New York, NY: Alfred A. Knopf. ISBN 9780679454434.
  6. Bauer WD. From Mechanics to thermodynamics. 2021, released 6.7.98 little additions 11.7.98 https://www.overunity-theory.de/2ndlaw/2ndlaw.htm.
  7. Shannon, C. A Mathematical Theory of Communication. 1948 Bell System Tech. J. 27, 379–423.
  8. Jaynes ET, Rosenkrantz RD (Ed.), 1983 Papers on Probability, Statistics and Statistical Physics, Reidel Publishing Company, Dordrecht.
  9. Kesavan H.K. (2008) Jaynes’ Maximum Entropy Principle. In: Floudas C., Pardalos P. (eds) Encyclopedia of Optimization. Springer, Boston, MA. [CrossRef]
  10. Caticha A. Entropic Dynamics. 2015 Entropy 17, 6110-6128.
  11. Caticha A. Entropic Inference and the Foundations of Physics. Invited monograph published by Brazilian Chapter of the International Society for Bayesian Analysis. 2012 ISBrA Sao Paulo, Brazil.
  12. Cafaro C, 2008 The Information Geometry of Chaos. VDM Verlag Dr. Mueller e.K.
  13. Caticha, Ariel. 2008. From Inference to Physics. The 28th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering. Boraceia Beach, Sao Paulo, Brazil. http://arxiv.org/abs/0808.1260v1.
  14. Frieden, B. Roy. 2004. Science from Fisher Information: A Unification. Cambridge, UK: Cambridge University Press. ISBN 0-521-00911-1.
  15. Shore, J.E., Johnson, R.W. 1980. “Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy”. IEEE Transactions on Information Theory, No. 26: 26-37.
  16. Shore, J.E. Johnson, R.W. 1981. Properties of Cross-Entropy Minimization. IEEE Transactions on Information Theory, No. IT-27: 472-482.
  17. Shore, J.E. 1986. “Relative entropy, probabilistic inference”. Uncertainty in Artificial Intelligence, AI. L.N. Kanal, J.F. Lemmer (Eds.), North-Holland, Amsterdam. 211-215.
  18. Astakhov K. 2009. Chapter 6: Methods of Information Geometry in Computational System Biology (Consistency between Chemical and Biological Evolution). Biomedical Informatics, Methods in Molecular Biology, No. 569:115-127. Humana Press. [CrossRef]
  19. Wilczek F. Physics in 100 Years. 2015. [CrossRef]
  20. Prigogine, I. 1961. Introduction to Thermodynamics of Irreversible Processes. New York, NY: John Wiley.
  21. Prigogine I, Stengers I. 1984, Order Out of Chaos, Brooklyn, NY: Bantam; ISBN: 0553343637.
  22. Kestin, Joseph. 1976. The Second Law of Thermodynamics. Hoboken, NJ: John Wiley & Sons Inc.
  23. Ho, Mae-Wan. 1998 The Rainbow and the Worm, the Physics of Organisms. River Edge, NJ: World Scientific. ISBN 981-02-4813-X.
  24. Kullback S. 1968, Information Theory and Statistics. Dover, New York.
  25. Rovelli C. Meaning = Information + Evolution. arXiv:1611.02420 [physics.hist-ph] doi.org/10.48550.
  26. Summers RL, Experiences in the Biocontinuum: A New Foundation for Living Systems. 2020; Cambridge Scholars Publishing. Newcastle upon Tyne, UK, ISBN (10): 1-5275-5547-X, ISBN (13): 978-1-5275-5547-1.
  27. Friston K. “The free-energy principle: A unified brain theory?” 2010 Nature Reviews Neuroscience. 11:12738.
  28. Karev, G. Replicator Equations and the Principle of Minimal Production of Information. 2010 Bulletin of Mathematical Biology. 72. 1124-42. [CrossRef]
  29. Karev, Georgy. 2010. Principle of Minimum Discrimination Information and Replica Dynamics. Entropy. 12. [CrossRef]
  30. Summers RL. Entropic Dynamics in a Theoretical Framework for Biosystems. Entropy. 2023; 25(3):528. [CrossRef]
  31. Denbigh, Kenneth. 1981. The Principles of Chemical Equilibrium: With Applications in Chemistry and Chemical Engineering, 4th Edition. Cambridge, UK: Cambridge University Press.
  32. Yang CT and Song CS. Theory of Minimum Rate of Energy Dissipation. Journal of the Hydraulics Division, 105:7. [CrossRef]
  33. Walker, S.I.; Davies, P.C.W. 2013, The algorithmic origins of life. J. R. Soc. Interface, 10, 20120869. [CrossRef]
  34. Harper M. 2009,The replicator equation as an inference dynamic. arXiv:0911.1763.
  35. Baez, JC, Pollard, B.S. 2016, Relative Entropy in Biological Systems. Entropy 18(2): 46-52.

Bio.

Dr Richard L. Summers is a Distinguished Guyton Professor and prior Vice Chancellor for Research at the University of Mississippi Medical Center and previously was the lead scientist for the NASA Digital Astronaut Project. He is the recipient of the prestigious Sebetia Ter International Award for Biomedical Research and author of the influential book Experiences in the Biocontinuum. Throughout his career as a noted theoretical biologist, Dr Summers has made significant contributions to our fundamental understanding of the complexity and functioning of living systems.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated