We explored the dynamics of the two interacting information systems. We show that for the Markovian marginal systems the driving force for information dynamics is determined by both the information landscape and information flux. While the information landscape can be used to construct the driving force to describe the equilibrium time reversible information system dynamics, the information flux can be used to describe the nonequilibrium time-irreversible behaviours of the information system dynamics. The information flux explicitly breaks the detailed balance and is a direct measure of the degree of the nonequilibriumness or time irreversibility. We further demonstrate that the mutual information rate between the two subsystems can be decomposed into the equilibrium time-reversible and nonequilibrium time-irreversible parts respectively. This decomposition of the mutual information rate (MIR) corresponds to the information landscape-flux decomposition explicitly when the two subsystems behave as Markov chains. Finally, we uncover the intimate relationship between the nonequilibrium thermodynamics in terms of the entropy production rates and the time-irreversible part of the mutual information rate. We found that this relationship and MIR decomposition still hold for the more general stationary and ergodic cases. We demonstrate the above features with two examples of the bivariate Markov chains.