Development of Tools to Assess Vaccine Literacy: A Scoping Review and Future Perspectives

1. Introduction

Vaccine Literacy (VL) is the ability to find, understand and judge vaccination-related information to make appropriate decisions about immunization. According to Ratzan [1], VL “is not simply knowledge about vaccines, but also developing a system with decreased complexity to communicate and offer vaccines”. There are different levels of VL. One is the personal level, which pertains to individual skills. Another is the organizational level, which includes the different degrees of complexity within an organization focused on communication and vaccine practice. Along with these two levels, there is a third larger, called population or community VL. Costantini et al [2] state that "VL is contingent on personal circumstances as well as the broader societal context...", while Budiyanti et al [3] report that "VL is a balance between individual, community and population skills in the complexity system". Other definitions include Badua’s et al [4]: VL represents a “process of providing vaccine information, building communication, and increasing people’s engagement about vaccines”. Finally, Zhang et al [5] refer to VL as “an important ‘endogenous driver’ of people’s vaccine choices, overcoming vaccine hesitancy and increasing vaccination rates”.

A recent review [6] has been published to collect, analyze, and summarize available definitions of VL, and to propose a comprehensive one. In summary, it has been proposed that VL is the degree to which people have the capacity to obtain and understand information regarding vaccination and related services. It entails ‘knowledge’, ‘motivation’, and ‘competencies’ to access, understand, and critically appraise and apply information about immunization, vaccines, and vaccination programs, at personal, organizational, and community levels.

Vaccinology is not only the science of vaccine development, is a very specific branch of medicine that deals with vaccines and immunization practices, including several biological and social sciences [7]. In fact, VL is linked to health literacy (HL), but the two realms overlap only partially. Competencies and knowledge about vaccines are unique: even individuals with varying levels of HL may lack the necessary skills pertaining to vaccination. Furthermore, VL is connected more than HL to a phenomenon strictly related to vaccination, which is Vaccine Hesitancy (VH).

VH is defined as an attitude of postponing or refusing vaccines despite their availability. Different VH models have been proposed. To the initial ‘3Cs’ model [8] (including individuals’ confidence, complacency and convenience toward vaccination) additional factors have been added to provide a better explanation to VH in the context of a complex social system showing evolving concerns towards vaccines [9]. Other attitudes from the personal and psychological perspectives have been included in expanded models, like the ‘5Cs’ (comprising calculation and collective responsibility, in addition to confidence, complacency, and convenience) [10], and the ‘7Cs’ adding other two factors (compliance and conspiracy) [11]. VL encompasses these elements in its 'motivation'-related dimension, but it also includes other dimensions, like knowledge and competencies, that are not part of the psychological determinants.

A meta-analysis [12], two systematic [5,13] and a scoping review [14] have been conducted recently on VL, including related assessing tools, replying to questions about the VL levels in the population, and its determinants and outcomes, before and amidst the Covid-19 pandemic. Various instruments have been developed to assess VL, although they may be considered limited by the number and complexity of influencing factors [5]. Therefore, there is a need to develop new, more comprehensive tools for a more accurate assessment of VL, which can comprehensively consider most of the influencing factors.

The objective of this study is to provide an overview of existing research on VL and its measures, as a kind of state of the art of this specific topic of preventive medicine. A second objective, based on selected literature and our previous work, is to deepen the understanding of the different visible or latent factors underlying VL and to suggest a framework as a basis for the development and validation of future tools.

2. Materials and Methods

The methodology used for this paper can be divided into three main steps.

First, we updated our previous review [14], particularly by looking at the tools and statistical methods used. Second, we performed a post-hoc evaluation of data from a mid-2020 survey [15] through mediation and factor analysis, where variables were re-labeled to improve the understanding of the relationships between them. Third, we elaborated a theoretical framework based on the literature review, and on a backward citation search.

Based on these steps, we have proposed a process for the development and validation of new VL tools-

2.1. Review Update

This review was based on our recent research [14], conducted according to Peters’ et al. methodology and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [16,17]. Findings from that scoping review were supplemented by a new search, using the same strategy and databases (PubMed/ MEDLINE, Embase, Web of Science, Cinahl, Scopus, and Psycinfo)), collecting publications from December 1st 2022 to December 1st 2023, as previous search had been conducted from inception to December 1st 2022. The following search string was used in Pubmed: "vaccine literacy" OR "vaccination literacy" OR "vaccination health literacy" OR "vaccine health literacy”, while for Embase, Cinahl, Scopus, and Psycinfo the search string using the following terms: “vaccin* literacy” OR “vaccin* health literacy”. To be included in this review studies should have: described a tool / instrument / questionnaire / measure explicitly assessing VL, reporting a VL score, and at least one determinant or outcome of VL. For determinant we considered any sociodemographic variables that could influence VL. By mediators, we considered any variable that can be influenced by VL, particularly beliefs, attitudes, behaviors, knowledge of participants, through which specific outcomes or effects occur (such as VH or acceptance, and vaccine uptake). By mediators, we mean any variable explicitly declared as such by the author and that influences the relationship between VL and its respective outcomes. A data charting form with the following elements was drafted: Author, reference and year of publication - title, country of the study and kind of population enrolled - number of subjects enrolled, gender and age - design and time period of the study – VL determinants (moderators) - VL scales used – mediators (when reported) - outcomes (dependant variables) - main findings. Data extraction was performed by two independent reviewers, then results were compared.

2.2. Statistical and Post-Hoc Analyses

For this review, statistical analysis was aimed at comparing descriptively the populations being studied, and results, such as demographics and VL scores, with those reported in the studies previously published [14].

In addition, we carried out a post-hoc analysis from the dataset of our survey conducted in 2020 in which a sample of 885 individuals of the general population was recruited to fill out an online questionnaire to evaluate VL levels on Covid-19 [15]. In the perspective of development of future tools, we explored the mediating role of VL between demographic antecedents and beliefs about general vaccination, and carried out a more in-depth factor analysis, which had not been done before, at 5% confidence level. Principal Component Analysis (PCA) was applied to determine a minimal number of items explaining a higher amount of variability, followed by Confirmatory Factor Analysis (CFA) used to confirm the adequacy of potential new scales. SPSS v27 software [18] was employed, together with the open source software Jamovi v2.4.11 to complement analyses with additional tests (like the generalized mediation model [19]). The use of more software also allowed to verify the consistency between findings.

To perform the post-hoc analysis, beliefs regarding the two statements included in the questionnaire (Appendix A) about the safety of vaccines (‘I am not favorable to vaccines because they are unsafe') and need to be vaccinated (‘There is no need to vaccinate as natural immunity exists’) has been taken as measures of ‘confidence’ and ‘complacency’, respectively, and evaluated through a four-item scale, while the answer to the question 'Do you want to pay a fee to be vaccinated?' as been considered as a measure of 'convenience'’ (evaluated through a nominal scale: possible replies: yes or no). Last seasonal flu vaccine received (self-reported) was considered as main outcome, while the question about intention to be vaccinated against Covid-19 has been taken as an attitude, considering that at the time of the execution of the survey (mid-2020) the SARS-cov-2 vaccines were not yet available and it was even unclear if and when they would be approved.

2.3. Theoretical Framework

We created a theoretical framework based on the Health Literacy Skills Framework (HLSF) by Squiers et al [20], and the Paasche-Orlow [21]and Wolf model Moderators, including both proximal and distal determinants, alongside potential mediators—variables clarifying motives and mechanisms behind outcomes—were considered based on the literature search and according to psychological models, like the Health Belief Model (HBM) [22] and the Protection Motivation Theory (PMT) [23].

3. Results

3.1. Review Update

In addition to those selected in our previous review [14], we found a total of 367 papers on PubMed and the other databases, published from December 1st 2022 until November 30th 2023 out of which we selected those where VL was assessed using specific tools, as described in Methods. Of the 246 papers screened by title and abstract, 17 publications were selected for this review, as shown in Figure 1 (PRISMA diagram) and summarized in Table 1. Other publications found in the screened literature were devoted to the use of HL assessment instruments, which were analyzed, and some commented, but were not considered for review, as not specifically dedicated to VL.

Table 1. Selected studies published in 2023 using VL tools.

Table 1. Selected studies published in 2023 using VL tools.

Diverse populations were included in the 17 papers selected in the present review (Table 1). Samples’ size varied from 133 to 12,586. Participants were distributed in different classes of age, but they were mostly young female adults. Nine publications out of the 17 selected for the present review confirmed the association between VL and vaccine acceptance or intention to be vaccinated (Table 1).

The scales employed in the selected articles were mainly modified versions of those used to measure HL chronic patients [41], and founded on the three-level HL model proposed by Nutbeam [42], including functional, interactive (or communicative) and critical levels. The HLVa-IT tool (Vaccine HL for adults in Italian) [43], later translated into English (HLVa) aims at measuring VL levels associated with routine vaccination of adulthood, including five questions assessing functional VL (FUVL), in addition to five and four items for interactive and critical skills, respectively. The functional sub-scale engages the semantic system, whereas the interactive and critical subscales regard more advanced cognitive efforts. The interactive-critical component of literacy may include also abilities regarding eHealth and AI-based approaches. A measure with the same construct has also been used in parents of children [44,45]. In all these tools, answers are rated on a forced 4-point Likert scale: a mean (± SD) score is calculated (range 1 to 4), treated as continuous, a higher value corresponding to a higher VL level. The validation process of HLVa in different languages is described elsewhere [14]. A study of the tool translated and adapted into Chinese has been published recently [38].

Based on the same construct, a measure was also developed and largely used to assess specifically Covid-19 VL (Covid-19-VLS), where the interactive and critical subscales have been merged and identified as interactive-critical VL (ICVL) and the total number of questions reduced from 14 to 12, to lessen redundancy. Covid-19-VLS also includes items to assess other variables (mediating factors, like beliefs and attitudes), and behaviors, like vaccine Covid-19 vaccine intention and flu vaccine acceptance and self-reported uptake [14]. Two versions of this tool were used, before and after the SARS-CoV-2 vaccines authorization and availability. Both measures are reported in Appendix A. During the pandemic, other tools were developed that utilized the HLVa construct. However, some of these instruments had a reduced number of items or employed different scoring methods than the original instructions. As a result, making descriptive comparisons became challenging.

3.2. Outcomes

Regarding the correlation between VL and vaccine acceptance, in two web surveys [33,34] Maneesriwongul et al have explored Thai parental attitudes and VL about Covid-19 vaccination. While nearly all parents of children under five years of age received their own vaccine, only 45% intended on vaccinating their sons. Factors influencing vaccine intention included parental age, attitudes, advice from healthcare professionals, VL and belief in vaccine effectiveness. Concerns over vaccine side effects and a need for more information contributed to VH. In the other study, out of 542 parents of children aged 5–11, 59% intended to vaccinate their child: Influencing factors included child age, parents’ education, VL, and positive beliefs on the Covid-19 vaccine. The parents’ VL, in terms of interactive/critical literacy skills, was among the most significant factors influencing parents’ intention to vaccinate their children. As for the VL score observed in these two studies, FUVL was between 2.67±0.69 and 2.8±0.71, and ICVL between 3.31 ±0.51 and 3.3 ±0.56 (score range 1-4). These values are consistent with those reported by the same Authors in the validation study of the VL tool conducted on the Thai general population [46].

On the contrary, other studies - like Iskender’s et al [30] and Bektas et al [26] - did not confirm a positive association between the VL’s ability and attitudes to get vaccinated against Covid-19, although in both studies it was not clear which variable was considered antecedent and which mediator (Table 1). The first survey examined the effect of Covid-19 VL on the attitudes towards vaccination among Turkish university students, showing that the level of parental education, field of study, and socio-economic status were associated with VL and vaccine intention. The correlation between VL and attitudes towards the vaccine was low. The other study [26] investigated the impact of Turkish parents' VH and Covid-19 VL on attitudes towards vaccinating their children during the pandemic. Parents' health was a factor in their attitude towards vaccination of children, which was low, due to a lack of confidence in the vaccine and a perception that it was risky, while VL did not affect their attitudes toward vaccination.

An average VL score has been calculated from seven out of the 17 selected studies: the FUVL score was 2.50 ± 0.33, whereas the ICVL was 3.03 ± 0.26. These values have a similarity to those that were reported in our previous scoping review (2.83 ± 0.25 and 2.92 ± 0.42, respectively) [14], despite lower functional levels, which is difficult to interpret, due to the inhomogeneity of the studies. Moreover, the VL scores reported in the other studies are not comparable, as not calculated as per the original instrument’s instructions [43]: in some studies the score range considered was 1 to 5, rather than 1 to 4, or a summative score was used instead of the mean score (Tab 1).

The role of the different variables as mediators between VL and outcomes, was evaluated in three papers using VL tools [28,32,36]. Shon et al [36] investigated the relationship between flu VL, health beliefs, and influenza vaccination. VL was assessed through the concept of understanding information, by administering one single question: “Based on given information, do you feel sure about the best choice (vaccinated versus non-vaccinated) for you to prevent flu infection?”’ (response options were yes or no). It was shown that flu VL impacted both on flu vaccine uptake and health beliefs assessed by the Health Belief Model (HBM). Analysis confirmed a mediating effects of health beliefs (perceived benefits, severity and susceptibility) between VL and vaccination.

Again with regard to the mediating effects, Collini al [28] showed that confidence in flu vaccine (measured through the Vaccine Confidence Index - VCI) completely mediated the relationship between ICVL (assessed through HLVa) and the intention to get vaccinated, with significant effects observed in different population subgroups.

Similarly, using data from an online survey conducted in China, Lu et al [32] have described how the ‘‘3Cs’’ psychological antecedents of vaccinations (confidence, complacency, convenience) can mediate the relationship between VL (in particular ICVL) and VH assessed through a specific 10-items scale. Results also confirmed that higher VL is associated with lower hesitancy, and time-to-event analysis showed that participants with increased VH had a longer delay in vaccination.

3.3. Post-Hoc Analysis

3.3.1. Mediation

The post-hoc analysis of our 2020 survey data (N=885) [15] was conducted using a multi-mediation model [19] where the educational level of study participants, sex, being a healthcare provider, and class of age were introduced as covariates, together with FUVL and ICVL. The results showed a significant mediating effect of ICVL between the independent variable ‘education’ and each of the ‘3Cs’ (accounting for 32%, 25%, and 57% of the total effect for confidence, complacency, and convenience, respectively, P between =,002 and < .001, bootstrapped C.I. 95%, 1,000 samples), while the only significant indirect effect of FUVL was between education and confidence, accounting only for 11% of the total effect (P= .032). Introducing in the model the ICVL items one by one, #10 and #12 were always those most significantly mediating between education and the ‘3Cs’, while among functional questions, #1 and #2 (ref. to Appendix A) showed the highest mediating effects. For example, for ‘confidence’, questions #10 and #12 both accounted for 20% of the total effect (P= .004, and P= .005, respectively).

For their part, all the ‘‘3Cs’’ components proved to be significant, although partial mediators of ICVL toward the outcome ‘seasonal flu vaccine uptake’ (confidence 37%, complacency 26%, and convenience 33% of the total effect, P between = .002 and < .001) while the only significant mediator of FUVL was confidence (14%, P< .025). The introduction of the different covariates in the model did not substantially change the results, with the exception of a moderator effect of increasing age on the relationship between FUVL and confidence toward influenza vaccine uptake (P=.014) which is in agreement with the idea that older individuals may develop more confidence than young people because of the increased experience with this vaccination practice and recommendations [47]. Data is contained within the Supplementary Material.

3.3.2. Factor Analysis

From the same mid-2020 survey dataset [15], we have entered the 12 VL items in PCA. along with other variables placed between determinants and outcomes of the framework (Figure 2), namely, two items regarding beliefs and five about attitudes, with a total of 19 variables. The first two factors explained 38% of the total variability (initial Eigenvalues, based on values >1), and 35% after Varimax rotation (ref. to supplementary material). Iterative PCAs were conducted to reduce the pool of 19 to six items, based on the scree plots, loadings, communalities, and potential relevance to the various underlying dimensions. Before reduction, the first factor, including five items, corresponded to the VH ‘3Cs’ in addition to the intention to be vaccinated against Covid-19, which was loading on the same component, the second factor (eight items) corresponded to ICVL, and the third (four items) clearly represented FUVL, while the remaining items were more dispersed. The load of the item related to Covid-19 vaccine intention on the first factor support the choice of taking it as an attitude, more than an outcome.

After reduction to six items, the first two factors explained 63% of the total variability both before and after Varimax rotation, still maintaining an acceptable internal consistency of the dataset, assessed using McDonald’s ω. CFA showed an acceptable data-model fit [48], with CFI, SRMR and RMSEA values going from 0.919, 0.0525, and 0.0589, respectively (19 items), to 0.984, 0.0221, and 0.0653 (six items). The combination of these techniques has already been used in the field of vaccination [49,50]: we found it important to check if CFA could confirm the construct suggested by PCA, although the analysis was executed on the same population for both tests.

3.4. Theoretical Framework

The logical framework developed depicts the relationship between knowledge, motivation, and competencies incorporating them along with functional, interactive, and critical VL levels. We considered moderators (proximal and distal determinants), and possible mediators (i.e. variables that explain the reasons and mechanisms behind outcomes). These variables include communication, knowledge, beliefs, attitudes, behaviors, self-efficacy and competencies (Figure 2).

4. Discussion

Some studies included in this review support the notion of a negative association between VL and VH [28,32,33,34,36], These findings were contradicted by other investigations published during the same period [26,30]. The results correspond to those of two recently published systematic reviews, where 10 out of 13 papers reported a positive association between VL and vaccine acceptance [13], and 18 out of 21 publications showed an association between VL and acceptance, positive attitudes, or beliefs toward vaccination [5].

Since all published studies are cross-sectional online surveys with a one-time measurement of VL levels, it is difficult to infer precise causalities. Furthermore, their heterogeneity prevents comparisons in terms of methods used, results and variables that may have influenced interpretation. For all these reasons, interpreting the impact of the VL remains challenging, although data from current and previous literature are substantially more in favor of a relationship between higher VL levels and vaccine intention or acceptance, and allow considerations on the association of VL with other variables, its mediating role and prospects for future research. In particular, we proposed a path for developing new tools to assess VL, based on selected and previous publications, along with their different methodological approaches, and sustained by a post-hoc analysis of our previous data sets.

A meta-analysis has been published on the same subject [12] at the time when this paper was drafted, conducted on 18 selected studies most of which already reported in previous reviews. Results confirmed VL as a strong predictor of vaccination intention, while its association with vaccination status (vaccine uptake) was weaker, although it had been demonstrated in several publications [14]. In our opinion, these results fit the proposed theoretical framework, considering the positioning of VL in it, and the relevance of its indirect, as well as the direct effect on the results, as described above.

4.1. The Role of VL toward VH

Vaccination is a preventive measure primarily targeted at healthy individuals. It often involves assuming responsibility and making decisions for others, such as parents making decisions for their children. [6]. Because of this, predictors of vaccine acceptance (like educational levels, socio-economic status, comorbidities, etc) may differ from those of other health behaviors, and personal reasons for getting vaccinated may be dissimilar to those that determine people’s decision about healthcare. Similarly, the skills required to navigate, comprehend, assess, and utilize information regarding immunization may vary from the skills needed for other health issues [54,55].

However, there is an ongoing debate regarding the extent to which VL directly or indirectly influence health behavior, such as vaccine intention, acceptance or uptake, and the subsequent health outcomes. While a positive impact on VH is frequently reported by VL, its specific role in this relationship remains unclear [5]. Aharon et al [44] observed that Israeli parents’ intention toward vaccination was not correlated to VL: those with higher skills were at significant greater risk of not vaccinating their children. Another study conducted in the same country and population during the Covid-19 pandemic revealed that parents who planned to vaccinate their children had slightly higher VL levels compared to hesitant parents. However, this difference was not statistically significant (P = .06). Additionally, VL was not correlated with Covid-19 vaccine intention, while all VH “5Cs” factors were. Similar findings have also recently been confirmed in studies selected for this review. For instance, Bektas et al conducted research on Turkish parents and reached similar conclusions [26].

In contrast, direct or independent association between higher VL levels and vaccine acceptance has been shown in several papers and reported in different reviews [5,13,56]. This association has also been confirmed by some studies published in 2023 and selected here, conducted in the general population of young adults [39], students [29] and parents [33,34,37].

4.2. The HL and VL Mediating Role

The mediating role of HL has been assessed in various relationships such as between education and health outcomes [57], socio-economic and health status [58], and, more recently, in the pandemic context, between health care system distrust and VH [59], and between personal antecedents and vaccine confidence [60]. Other studies have investigated effects of HL and VL when mediated by other variables, although with different objectives and methods (Figure 3):

Using a 11-items tool derived from HLS-EU-Q47, including general questions about immunization (like “Understand why you need vaccinations”), Jiang et al [61] showed that the relationship between perceived HL and Covid-19 vaccine acceptance was completely mediated by attitudes toward general vaccination and self-efficacy of Covid-19 vaccine. The path from determinants to vaccine acceptance has been illustrated by Hurstak et al [60,62]. Authors showed in a selected population of urban adults, using a functional HL tool (Touchscreen Technology - LiTT) and an adapted vaccine confidence scale, that HL mediated the relationship between some demographic variables and vaccine confidence, which in turn mediated the relationship between HL and Covid-19 vaccine acceptance. According to Lu et al [23], using Covid-19-VLS in the Chinese general population, all the ‘3Cs’ psychological antecedents played a significant role in mediating VL with VH, accounting for 66% and 95% of the total effect of FUVL and ICVL, respectively.

Using the same tool to check mediating effects on our previous data series [15] (unpublished data) we found that VL (in particular ICVL) significantly mediated in the Italian general population the relationship between demographic variables, such as education level, and positive beliefs about vaccination, assumed as confidence, complacency and convenience. In their turn, beliefs partially, but significantly mediated the relationship between VL (particularly ICVL) and flu vaccine uptake.

Using a VL single item nominal tool Shon et al [36] demonstrated the mediating effects of health beliefs (Health Belief Model’s perceived benefits, severity, and susceptibility) between flu VL and flu vaccine acceptance in students, although the literacy of influenza vaccines also directly improved the vaccination behavior of individuals without any health-mediating effects of belief. Finally, Collini et al found that vaccine confidence (assessed by the same VCI scale used by Hurstak) completely mediated the relationship between ICVL (assessed through HLVa) and the intention of nursing home personnel to get vaccinated against flu [28].

Comparing different mediation models is challenging and should be interpreted carefully. However, these data show that VL can significantly influence vaccine acceptance both directly and indirectly. Clearly, the HL and VL roles may vary depending on the tools used. The last above Authors had previously shown that there was no significant correlation of vaccine uptake with HL when evaluated by a general functional tool (Imeter), and the VCI score and the HL score were only weakly correlated [63]. This was confirmed in another survey using the same functional tool, where an association was found between vaccine intention and the VCI score, but not with the HL score, and the relation between vaccine intention and the HL score was not statistically significant [64]. The same, HL did not affect the likelihood of influenza vaccination uptake and did not mediate the relationship of any independent socio-demographic variable with flu vaccination in a population of high-risk individuals, using the 6-items European Health Literacy Survey Questionnaire (HLS-EU-Q6) not including specific items on immunization [55,65].

In summary, available data indicates that specific VL assessment - especially ICVL - instead of general HL can better contribute to analyzing VH predictors. Indeed, in addition to its conceptual definition, VL can be explained by its ‘location', as it sits at the intersection between antecedents (moderators) and intermediate variables (mediators), partially overlapping these last (Figure 2). The greater the involvement of VL tools in the mediation area, especially concerning beliefs, attitudes, motivations, and self-abilities, the more significant impact VL can have as a mediator and driver to promoting health behaviors and outcomes, such as vaccine acceptance, i.e. the degree to which individuals accept or refuse vaccination [66], and vaccine uptake, i.e. the proportion of individuals who have actually received a vaccine.

4.3. Current VL Tools

The VL tools developed so far are intended for the assessment of the personal VL of the adult general population, although some have been adapted to selected adult populations.

HL and VL have received growing attention through research during the Covid-19 outbreak, where new measures have been proposed assessing coronavirus literacy and exploring attitudes and behaviors. Several tools were developed or adapted to assess HL levels from the onset of the pandemic, such as CoV-eHEALS [67], HLS-Covid-Q22 [68], and an updated version of the European Health Literacy Questionnaire (HLS19-Q47) [69] including four vaccination-related questions to assess vaccine HL. In other surveys, HL was measured using the 12-item short version of HLS-EU-Q, integrated with three vaccine-related items [70]. As mentioned, following the first tools for the assessment of VL (HLVa), specific instruments were developed to conduct studies during the coronavirus outbreak, to accurately assess VL levels, such as the Covid-19-VLS, including functional and interactive-critical subscales, in addition to measures for evaluating other variables, such as beliefs, attitudes and behaviors toward Covid-19 vaccines, despite in this instrument the evaluation of the different variables is calculated by different rating methods.

Functional literacy refers to the use of semantic and cognitive abilities like reading, writing, and knowledge of medical terms, and mathematics [71]. The quantification of these abilities is possible using performance-based tools. Inversely, self-report measures typically evaluate the psychological aspects that underlie components like motivation, beliefs, attitudes, and the ability to engage with information and make decisions (listening, speaking, interpreting). Standardized questions are used in objective assessment to measure underlying characteristics, while subjective measurement involves, typically on Likert scales, people self-reporting to questions on their experiences about health, although it is challenging to establish a connection between persons’ responses and their actual skill [72].

Performance-based tools appear more suitable for estimating individuals’ skills in the health care domain, while self-reported measures are better for assessing individuals’ attitudes and knowledge beyond reading and numeracy, such as understanding the value of vaccination. Considering the domains relevant to VL (disease prevention and health promotion [73]), the use of subjective VL tools, such as HLVa, and derivative measures, seem appropriate for that scope, as they entail items related to motivation and competencies. Their construct has been validated in the general population of different regions. PCA, and Exploratory Factor Analysis (EFA), and CFA, were used to extract the latent factors defining their construct. Through parallel analysis, all have identified two separate components underlying the FUVL and ICVL items, explaining high and comparable percentages of the total variance, and significantly similar factor loadings [56].

It has been pointed out that some latent factors might be underestimated in current VL tools [5]. However. similar to HL, VL is a latent construct by definition [74]: reliable measurement scales can be constructed and validated to capture VL effectively, although the results may represent aspects of VL without claiming to encompass its entirety. Indeed, we believe that despite the limitations of the current VL tools, and even if the surveys conducted so far are mostly cross-sectional and carried out primarily in the context of the pandemic, the accumulated experience remains important. The Covid-19 pandemic has likely influenced public sentiment towards the prevention of viral diseases, leading to long-term impacts on the way the general population perceives all communicable diseases. The pandemic experience will in any case affect VL with regard to other vaccines, at least in the near future. Thus, although the experience of VL tools used mainly during the Covid-19 outbreak can be considered limited, it provides a relevant reference for future research [58].

4.4. Future Perspectives

As for HL [74], new methods for VL measurement should explicitly refer to the domains outlined in relevant conceptual frameworks. In addition, to address the limitations of current measures, and align them as much as possible with the most recent definitions of VL, the construct of new instruments should contain and integrate items related to all the three components (motivation, knowledge, competencies), thus reducing the risk of underestimating latent factors. We attempted to determine an effective approach for incorporating them along with the three VL levels (functional, interactive, and critical) within the Health Literacy Skills Framework (HLSF), as illustrated in Figure 2. The framework, proposed by Squiers [20] was used for our analysis together with the Paasche-Orlow and Wolf model [21]. We considered moderators (proximal and distal determinants), and possible mediators (i.e. variables that explain the reasons and mechanisms behind outcomes). These variables include communication, knowledge, beliefs, attitudes, behaviors, self-efficacy and competencies, incorporating concepts from various psychological theories, such as, among others, the Health Belief Model (HBM) [36], and the Protection Motivation Theory (PMT) [52]. We referred to tools designed for assessing these variables, relevant in the vaccination field, also taking into account the mentioned VH models (Table 2). Based on this, we have endeavored to outline potential new VL tools.

Motivation: there are different definitions for motivation [75]; according to the American Psychological Association) [76], motivation refers to “a person’s willingness to exert physical or mental effort in pursuit of a goal or outcome”. It is a dynamic concept, resulting of internal and external inputs that lead to decisions and behaviors [75]. Motivation encompasses attitudes, which includes beliefs, emotions, and evaluations. In clinical settings there are various tools for measuring motivation [75]. Considering the consistency of the results reported so far from studies using HLVa and derived measures (Covid-19-VLS), the VL scales entailed in these tools appear suitable for assessing motivation also for future research, in particular in the interactive-critical subscale. Redundancy of questions identified during the validation process was addressed through factor analysis [77] reducing the number of questions from 14 to 12 overall (four for FUVL and eight ICVL) [15]. As reported above, additional analyses can be performed to further reduce the number of questions to lessen as much as possible their total number and facilitate their filling in. It is also possible to design VL tools to assess motivation by adapting items from the PMT, which explains how people respond to fear-evoking or threatening messages, or from the HBM, which explains and predicts health behaviors by examining the attitudes and beliefs, as previously demonstrated by others [78]. The mediating role of beliefs, evaluated by the HBM, in the relationship between VL and flu vaccination has been studied in one of the selected publications [36]. Noteworthy, the papers on behavorial change models cited in Table 2 represent just examples of a very extensive literature.

Knowledge: is the state of being familiar with something or aware of its existence, usually resulting from experience or study (i.e. information learned) [76] . It is part of the conceptual VL definition and a key mediating component in the VL framework. Procedural knowledge, which involves understanding how to carry out specific tasks or actions, serves as the basis for ICVL skills and fluid cognitive abilities, whereas crystallized abilities, including generalized knowledge and vocabulary, are typically more associated with functional skills, particularly in the elderly population [71]. Understanding the relation of VL to basic cognitive abilities is important since research has shown that both general intellectual abilities and literacy are related to health [79]. While IQ tests can provide a broad assessment of cognitive abilities, adding psychological assessments to the tools designed for the general population can be challenging. Evaluating the level of crystallized knowledge through a vaccine quiz can offer a straightforward, relevant, and efficient performance-based method. This approach has been previously employed for validating the theoretical construct of HLVa-IT, where the quiz was administered together with the VL questionnaire [77]. In addition, vaccines quizzes can be structured to objectively evaluate the individual’s functional reading and understanding skills, when administered through face-to-face interviews or by paper-and-pencil. Regarding online surveys, there is an inherent risk that individuals may seek assistance or refer to external sources, which can potentially inflate their scores on assessments. However, measures can be taken to reduce this risk, such as underlining the anonymity of the survey, or by designing the questionnaire in a linear, one-directional flow where respondents can only move forward, and by including control or confirmation questions in the final page.

The content of the scales can be adapted over time according to the vaccines that will become available in the future, although addressing the general knowledge about vaccination and on the most common routine vaccines (D, T, Polio, Influenza) would reduce disparities. Items can be selected from vaccine scales available in the literature [80], and online resources provided by academic [81] and international institutions [82]. Common questions on the knowledge of vaccines and diseases should be identified and used for comparability purposes, while taking into account cultural and socio-economic differences between populations.

Table 2. Psychological theories explaining behaviors toward vaccination and related assessment measures, usually rated through odd-points scales. Different examples of items reported in the literature (for influenza, Covid-19, and HPV) are proposed for each component of the models, to be possibly used or adapted for building new VL tools to explore motivation and competencies, while knowledge can be assessed by performance-based measures, administering standardized stimuli.

§ = there are several scales to assess VH [10]; a frequently adopted measure is VCI [85,86] - Flu example [28,63] assessed on 4-point Likert scale - VCI = [ (A1+A2+A3+A4) / 4] / [(B1+B2+B3+B4) / 4]. (A1) Flu is a serious illness - (A2) Flu vaccine is effective - (A3) HCWs must get vaccinated - (A4) By getting vaccinated I can protect people close to me; (B1) It is better to contract flu than to get the vaccination - (B2) Flu vaccines have serious side effects - (B3) Vaccine can cause flu - (B4) Opposed to flu vaccinaton.

Table 2. Psychological theories explaining behaviors toward vaccination and related assessment measures, usually rated through odd-points scales. Different examples of items reported in the literature (for influenza, Covid-19, and HPV) are proposed for each component of the models, to be possibly used or adapted for building new VL tools to explore motivation and competencies, while knowledge can be assessed by performance-based measures, administering standardized stimuli.

§ = there are several scales to assess VH [10]; a frequently adopted measure is VCI [85,86] - Flu example [28,63] assessed on 4-point Likert scale - VCI = [ (A1+A2+A3+A4) / 4] / [(B1+B2+B3+B4) / 4]. (A1) Flu is a serious illness - (A2) Flu vaccine is effective - (A3) HCWs must get vaccinated - (A4) By getting vaccinated I can protect people close to me; (B1) It is better to contract flu than to get the vaccination - (B2) Flu vaccines have serious side effects - (B3) Vaccine can cause flu - (B4) Opposed to flu vaccinaton.

Competencies: can be viewed as a set of knowledge, skills, capabilities (abilities), and behaviors that contribute to the individual’s performance [87]. Given that knowledge on vaccines can be evaluated as described above, and that skills and abilities can be assessed through the existing VL scales (such as the interactive-critical sub-scale of HLVa), it is suggested to complete the assessment of competencies by incorporating standardized items that evaluate vaccination behavior (such as vaccines received), together with the educational level of respondents. This suggestion is based on the general understanding that individuals with higher education levels are more likely to adopt healthy behaviors [88]. Education plays a crucial role in providing individuals with knowledge and skills required to develop competencies in various domains such as health, although HL is not solely dependent on educational levels, and these competencies can also be developed through other means in addition to education. Selected items from self-efficacy scales [53] specifically developed and adapted to the vaccination field may be useful in completing the assessment of competencies. However, it's important to underline that competencies refer to the actual knowledge and skills possessed by an individual, while self-efficacy relates to an individual's belief in their capability to use their competencies effectively [76].

In the literature, composite instruments are reported combining scales to evaluate different variables related to vaccination [10,89], while creating multidimensional composite VL tools is challenging because of the complexity of influencing factors, and the many concepts involved. Elements related to psychological content, such as motivation, beliefs, and attitudes, are often included in a variety of other tools usually administered together with HL or VL questionnaires in conducting investigations. Incorporating some of these elements in a single VL tool would standardize responses, making comparisons easier. Our proposed approach involves selecting a few items for each VL dimension and incorporating additional questions related to beliefs, attitudes, and behaviors, in addition to knowledge. The selection of these elements can be guided by meaningful grouping [90], reliability and factor analyses, in addition to specific statistical methods to handle categorical variables, like the two-stage path analysis [91]. These methods can ensure the construct validity of composite instruments, despite the reduction in the number of elements [73].

Indeed, Covid-19-VLS, frequently utilized in recent studies [5,13,56], can already be considered a composite VL measure. In addition to the VL subscales, it includes questions related to beliefs and attitudes underlying confidence, complacency, and convenience, in addition to coronavirus vaccines intention, and behaviors such as flu vaccine uptake (Appendix A). Although a reduction in VL items was made compared to HLVa, we were able to further reduce their dimensionality by applying PCA as a method of data reduction. This exercise was done also by considering the results of the mediation analysis, balancing the weight of the single VL items, with the purpose to ensure the use of similar elements in future research. This involves including additional questions on knowledge and skills to new assessment scales, maintaining an acceptable total number of items in order to balance the length of the questionnaire with people's willingness to participate in the surveys. However, the decision on how many items to use should consider the trade-off between maintaining significant factors, reliability, and data interpretation. This approach will also be useful in addressing a limitation of the current VL instruments, related to a possible underestimation of their specific dimensions [6]. In fact, the current VL instruments are derived from tools that were originally developed to assess HL in other areas of medicine (chronic patients), although they have been validated in various languages and cultural settings [14]

In summary, using similar metrics on all elements studied, at the same time balancing them, and considering their association with each other (i.e. VL linked to beliefs and attitudes, linked to outcomes), a composite VL score can be sought for future assessments. In addition to measuring separately each scale included in a multidimensional framework - useful to observe the correlations and the mediating role of the variables between them -, adding a standardized combined index would allow simplified representation and easier interpretation of results, as well as improve statistical power [92]. Moreover, including self-reported and performance-based elements in the same tool would facilitate a better assessment of the overall individual VL levels, without the need for additional tests. It has been shown that the combination of the results obtained using at the same time performance-based measures of functional HL and self-assessed measures of general HL may results in an increase in sensitivity (i.e. the identification of people with low HL skills) and improve the understanding of the relationship between HL and its antecedents [93]. It is reasonable to assume that the same can be valid for VL.

4.5. Specific VL Measures

Questions included in new VL tools should be adapted to the specific medical specialty or field for which the measure will be intended. As opposed to HL where there is a huge proliferation of measures [72], the number of tools for VL is limited. Therefore, for HL tools developed for several specific contexts and populations outside of pandemic emergencies, ideally a similar approach should be taken for VL. In addition, it will be important to develop dedicated tools for evaluating VL in different medical areas, such as routine vaccination for children, for patients and other specific categories (e.g., healthcare workers and travelers), as well as for unique psycho-physiological situations, for instance pregnancy, although existing VL tools have been used in some of them [44,94,95,96,97]. Moreover, in the future, vaccine applications will extend beyond the prevention of infectious disease. For instance, mRNA and siRNA techniques hold potential in various healthcare areas, including oncology and diseases with genetic components [98]. Therefore, the development of future VL instruments will extend beyond the specific area of communicable diseases.

Adolescents are another critical area for future research. The pandemic has had many negative effects on teenagers, especially in low- and middle-income countries, and vaccine coverage rates against SARS-CoV-2 in younger age groups were insufficient even in developed realms [99]. The controversial nature of coronavirus vaccination has exacerbated the pressure on parents who make decisions about their sons’ immunization. Yet, in recommending vaccines, it is important to consider not only parents’ attitudes to increase uptake, but also adolescents’ awareness of the infective risks and their knowledge of self-consent rules, and the relevance of taking part in vaccination decisions [100]. These aspects should be worth exploring through the development of specific VL measures. Over 40 tools are available for HL assessment in adolescents aged 10 to 7 years [101], but none for VL, so far.

The available VL scales have shown good consistency in results across different countries, which has been proven not only comparing the average scores observed between populations, but also comparing factor analyses data [14]. However, the cut off values have been set only arbitrarily, so far: VL has been defined as limited when the HLVa and Covid-19-VLS score is ≤2.5, or when belonging to the lower tertile of the average values observed in a given population. For example, in the mentioned 2020 survey the upper tertile bound corresponded to a score ≤2.50 for FUVL and ≤3.13 for ICVL, which allowed to define low-literate people, as already done by others [96,102]. While defining a universally applicable cut-off value, it would facilitate comparisons, using the lower tertile approach would more rigorously evaluate literacy levels according to local settings. In such a case, turning the score in a standardized one (observed value – mean of the sample / SD) could allow for meaningful comparisons.

Validation of future VL tools should ideally be performed internationally, to get results at the same time in more than one country, aiming to define a universally applicable threshold value, although also referring to local average scores to identify limited VL will remain. Future tools should be also proposed in prospective cohort and longitudinal studies for better understanding the causal relationship between VL and VH, and the relevance of the mediating role of VL. To improve homogeneity and comparability of populations new tools should be administered via the web in a standard manner trying to reduce as much as possible biases related to online surveys – like the social desirability bias -, using multi-item scales and combining self-report measures with other data sources, such as behavioral observations, as well as underlining anonymity. As mentioned, adding objective measures, such as about vaccine knowledge will be also helpful.

Finally, while a definition for organizational VL has been proposed [6] and it is already mentioned in literature [103], it is important to develop specific measures. Improving organizational VL is crucial for increasing vaccination rates, as healthcare organizations play a vital role in providing trustworthy and accessible information to the community. Specific instruments to evaluate organizational HL capacities have already been created and applied [104,105], It should be done the same for organizational VL, which should be the subject of dedicated research.

4.6. Limitations of This Review

Despite the use of various databases and attempts to be as comprehensive as possible, this review may not have identified all relevant recent articles in the literature, as the overall search strategy may have been biased toward public health. Searches of other databases may have resulted in other relevant publications. Furthermore, the search was conducted using only English terms, which possibly could have led to missing some studies. Limitations and the heterogeneity of online cross-sectional studies in terms of methods used, and reporting of results may have affected the interpretation of the data. Furthermore, due to heterogeneity of the results reported in the selected studies, the findings were addressed only descriptively.

Despite differences, most of the reviewed studies used the same scales validated in different populations and languages, which is a strength, considering the wide variety in rating scales of online questionnaires. Therefore, through associating various studies, we believe it has been possible to obtain a fairly accurate understanding of the current utilization of tools and the assessment of VL skills. However, this understanding remains largely descriptive, showcasing diverse values across different regions and populations. These variations are likely linked to methodological and/or local cultural differences.

Finally, in this review a post-hoc analysis of previous data sets and subsequent proposals for the development of future tools has been carried out, not to revise the results already published, which remain valid, but in view of the development of future tools, which purely reflect the point of view of the Authors and their experiences within VL domain. Comments and proposals from other research groups, hopefully numerous, will be welcome to broaden the discussion and progress on this important public health topic.

5. Conclusions

The papers selected for this review, confirm previous results, i.e that the relation between VL and VH is uneven, although the majority is in favor of a negative association. The role of other variables (like beliefs, attitudes, self-efficacy) as mediators of VL has also been investigated. As vaccination attempts to prevent infectious diseases in both the individuals and the community, the relevance of VL in the field of disease prevention and health promotion is evident. In this regard, the use of the current self-rated tools, such as HLVa and derivative measures seem appropriate to assess VL skills, despite some specific dimensions may result underestimated, due to the complexity of antecedents. Indeed, the VL construct depends on complex, distal and proximal determinants, represented by social-environmental and personal variables.

Based on the findings from selected studies and past publications, along with their design and statistical analysis, we have proposed a path to develop new tools for assessing VL. This proposal gains further support from our post-hoc analysis of our previous data series. Future research should be focused on developing measures including performance-based items, where possible, in accordance with the latest VL definitions, to objectively assess knowledge and competencies, in addition to the psychological components of motivation, beliefs, attitudes and self-efficacy. Such measures will facilitate further research about the mediating role of VL and its relationship with VH, as well as about unexplored aspects, such as the longitudinal evolution of VL in a given population in different contexts, and the application of research in the organizational literacy. The assessment of VL in various categories of population and groups of patients, in addition to healthcare workers, will also be needed.