Low back pain (LBP) is a leading cause of disability. It significantly impacts the patient’s quality of life, limits their daily living activities, and reduces work productivity. To reduce the burden of LBP, several pharmacological and non-pharmacological treatment options are available. This review summarizes the role of heat therapy in the management of LBP. First, we outline the common causes of LBP, then discuss the general mechanisms of heat therapy on (LB)P, and finally review the published evidence regarding the impact of heat therapy in patients with acute or chronic non-specific LBP. This review demonstrates that continuous, low-level heat therapy provides pain relief, improves muscular strength, and increases flexibility. Therefore, this effective, safe, easy-to-use, and cost-effective non-pharmacological pain relief option is still relevant in modern clinical practice.