There exists a compelling need to better understand the temporal dynamics of public sentiment towards COVID-19 vaccines in the US on a national and state-wise level for facilitating appropriate public policy applications. Our analysis of social media data from early February of 2021 and late March of 2021 shows that in spite of overall strength of positive sentiment, and increasing numbers of Americans being fully vaccinated, negative sentiment about COVID-19 vaccines still persists among sections of people who are hesitant towards the vaccine. In this study, we performed sentiment analytics on vaccine tweets, studied changes in public sentiment over time, conducted vaccination sentiment validation using actual vaccination data from the US CDC and Household Pulse Survey (HPS), explored influence of maturity of Twitter user-accounts and generated geographic mapping of sentiments by location of Twitter users. Furthermore, we leverage the emotion polarity based Public Sentiment Scenarios (PSS) framework which was developed for COVID-19 sentiment analytics, to systematically analyze directions for public policy processes to potentially improve the administration of vaccines. Application of the PSS framework provides important time sensitive insights for state and federal government agencies and associated organizations to better implement public policy processes for healthcare management, communication, transparency, motivation and societal operational policies such as social distancing. These insights are expected to contribute to processes that can expedite the vaccination program and move closer to the cherished herd immunity goal.