A popular account of the mixing patterns for the three generations of quarks and leptons is through the characters $\kappa$ of a finite group $G$. Here we introduce a $d$-dimensional Hilbert space with $d=cc(G)$, the number of conjugacy classes of $G$. Groups under consideration should follow two rules, (a) the character table contains both two- and three-dimensional representations with at least one of them faithful and (b) there are minimal informationally complete measurements under the action of a $d$-dimensional Pauli group over the characters of these representations. Groups with small $d$ that satisfy these rules coincide in a large part with viable ones derived so far for reproducing simultaneously the CKM (quark) and PNMS (lepton) mixing matrices. Groups leading to physical $CP$ violation are singled out.