Approximately half of head and neck squamous cell carcinoma (HNSCC) cases recur, and most of them recur within the first two years after treatment. Although it has been suggested that the interval to recurrence after radical treatment and prognosis in patients with HNSCC are associated, investigations are insufficient. In this study, patients diagnosed with HNSCC at Kyushu University Hospital were retrospectively analyzed (n = 500). A total of 234 patients experienced recurrence, with 110 and 124 patients experiencing early recurrence (ER) (recurrence within 2–6 months) and late recurrence (LR) (recurrence after 7 months), respectively. There were two independent risk factors for poor prognosis in the multivariate analyses: ER (hazard ratio [HR] = 3.200, 95% confidence interval [CI] = 1.570–6.521, p = 0.001) and no radiotherapy (HR = 0.374, 95% CI = 0.191–0.733, p = 0.004). In patients with recurrent HNSCC, a short interval to recurrence is a risk factor for poor prognosis and survival. This study demonstrated the prognostic value of ER in patients. During the selection of treatment for patients with recurrence, the recurrent area, initial treatment content, and strategy of changing salvage therapy depending on the recurrence status should be considered.