Experimental recordings of the collective activity of interacting spiking neurons exhibit random behavior and memory effects, thus the stochastic process modeling the spiking activity is expected to show some degree of time irreversibility. We use the thermodynamic formalism to build a framework, in the context of spike train statistics, to quantify the degree of irreversibility of any parametric maximum entropy measure under arbitrary constraints, and provide an explicit formula for the information entropy production of the inferred Markov maximum entropy process. We provide examples to illustrate our results and discuss the importance of time irreversibility for modeling the spike train statistics.