Background: Low birth weight (LBW) is a major public health problem in low-income countries. Infants born with LBW are at increased risk of neonatal mortality, have poorer school performance, and are more likely to develop respiratory and cardiovascular diseases in adulthood. Methods: An unmatched case–control study was conducted in five maternity wards in the Kenge Health Zone, involving 159 mother–newborn pairs. Data analysis was performed using STATA 18.0 software. Bivariate and multivariate analyses were used to determine the associations between independent variables and LBW. Results: Determinants of low birth weight were maternal short stature (height < 155 cm) (adjusted odds ratio (AOR): 3.90; 95% confidence interval (CI): 1.66–9.13), maternal non-indigenous Yaka ethnic group (AOR: 2.39; 95% CI: 1.01–5.66), having attended fewer antenatal care (ANC) sessions (AOR: 0.73; 95% CI: 0.53–0.99), and inadequate maternal dietary diversity (AOR: 3.97; 95% CI: 1.25–12.62). Conclusions: Increased education and awareness of dietary diversity as an indicator of dietary quality in women of childbearing age, especially during pregnancy, along with regular follow-up of antenatal care visits and behavior change communication targeting pregnant women, can help health professionals address dietary taboos and reduce the incidence of LBW.