The dynamics of many complex systems can be classified as ordered, chaotic, or critical. Order offers stability and robustness, while chaos allows for change and adaptability. Criticality, then, is often seen as a balance required by living systems at different scales. In classical models, however, criticality is only found near phase transitions, restricting the parameter space (and thus the likelihood) of critical dynamics, as most parameters yield ``undesirable'' solutions. Here we show that this limitation is due to the homogeneity built-in these models, i.e., all elements sharing parameter values. By exploring heterogeneous versions of archetypal models in physics and computer science, we observe critical dynamics in a broader range of parameters, and thus could be more common than previously thought.