Farming systems prevalent in sub-Saharan Africa are exposed and vulnerable to climate change due to their high dependence on rainfall. However, most studies have only estimated the impacts of climate change on agricultural productivity at a regional or national level. We add to this literature by focussing on the sub-national impacts. This study uses 30 years (1981–2011) of yield and weather data in Zambia and applies the Just and Pope model to determine how rainfall and temperature affect yield and yield variability of maize and beans at the national and subnational levels. Results show a negative impact of temperature rise on yield and a positive impact of rainfall rise on yield, above the current mean levels. These results differ by agro-ecological region. Worst-case-scenario predicted impacts using HadGEM-ES2 global circulation model show that major yield decreases (25% for maize and 34% for beans) by 2050 will be in region II and will be driven mainly by temperature increase offsetting the positive gains from rainfall increase. The model mainly under-predicts yield for maize and overpredicts yield for beans. These findings call for agro-ecological region-specific adaptation strategies and well-planned policy interventions to make agriculture more resilient to climate change.