COVID-19 vaccine hesitancy is considered responsible for the lower rate of acceptance of vaccines in many parts of the world. However, sources of this hesitancy are rooted in many social, political, and economic factors. This paper strives to find the most important variables in predicting the COVID-19 vaccination uptake. We introduce an explainable machine learning (ML) framework to understand the COVID-19 vaccination uptake around the world. To predict vaccination uptake, we have trained a random forest (RF) regression model using a number of sociodemographic and socioeconomic data. The traditional decision tree (DT) regression model is also implemented as the baseline model. We found that the RF model performed better than the DT model since RF is more robust to handle nonlinearity and multi-collinearity. Also, we have presented feature importance based on impurity measure, permutation, and Shapley values to provide the most significant unbiased features. It is found that electrification coverage and Gross Domestic Product are the strongest predictors for higher vaccination uptake, whereas the Fragile state index (FI) contributed to lower vaccination uptake. These findings suggest addressing issues that are found responsible for lower vaccination uptake to combat any future public health crisis.