Phosphorus (P) is the second-most important element after nitrogen that is required for plant growth. Although this element is abundant in most soils, it is rarely available in plant-accessible forms since most of it normally exists in soil in insoluble forms such as phosphates. In conventional agriculture, P is normally supplied as chemical fertilizer to satisfy plant P requirements. This, to a large extent, boosts plant production. However, chemical fertilizers are costly, have a huge carbon footprint, and are environmentally-unsustainable owing to the high energy requirements during their synthesis. Besides, P-containing agricultural run-offs contribute hugely to the eutrophication of water bodies and environmental degradation. Moreover, plants can consume only a small amount of chemically-supplied P since between 75 and 90% of this form of P normally get precipitated into complexes and rapidly become fixed in soil. These issues and concerns necessitate research into alternative and viable ways of supplying P to plants. Rhizobacteria have for decades been investigated in vivo and in planta as suitable tools in sustainable agriculture due to the plant-growth-promoting activities such as nutrients’ solubilization, nitrogen fixation, and production of phytohormones. Although a lot of research has been done on different nutrients-solubilizing rhizobacteria and their potential in sustainable agriculture, their mechanisms of action and prospects in sustainable agriculture remain to be fully understood. This review particularly focuses on the P solubilizing rhizobacteria and evaluates their diversity, mechanisms of action, and prospects in sustainable agriculture based on the present and future scenario of their application. Such information is useful in determining their potential and evaluating their prospects in promoting sustainable agricultural systems.