Most existing flexible count regression models allow only approximate inference. This work proposes a new framework to provide an exact and flexible alternative for modeling and simulating count data with various types of dispersion (equi-, under- and overdispersion). The new method, referred as “balanced discretization”, consists in discretizing continuous probability distributions while preserving expectations. It is easy to generate pseudo random variates from the resulting balanced discrete distribution since it has a simple stochastic representation in terms of the continuous distribution. For illustrative purposes, we have developed the family of balanced discrete gamma distributions which can model equi-, under- and overdispersed count data. This family of count distributions is appropriate for building flexible count regressionmodels because the expectation of the distribution has a simple expression in terms of the parameters of the distribution. Using the Jensen–Shannon divergence measure, we have shown that under equidispersion restriction, the family of balanced discrete gamma distributions is similar to the Poisson distribution. Based on this, we conjecture that while covering all types of dispersion, a count regression model based on the balanced discrete gamma distribution will allow recovering a near Poisson distribution model fit when the data is Poisson distributed.