When gravity is quantum, the point structure of space-time should be replaced by a non-commutative geometry. This is true even for quantum gravity in the infra-red. Using the octonions as space-time coordinates, we construct a pre-spacetime, pre-quantum Lagrangian dynamics. We show that the symmetries of this non-commutative space unify the standard model of particle physics with $SU(2)_R$ chiral gravity. The algebra of the octonionic space yields spinor states which can be identified with three generations of quarks and leptons. The geometry of the space implies quantisation of electric charge, and leads to a theoretical derivation of the mysterious mass ratios of quarks and the charged leptons. Quantum gravity is quantisation not only of the gravitational field, but also of the point structure of space-time.