Since all published studies are cross-sectional online surveys with a one-time measurement of VL levels, it is difficult to infer precise causalities. Furthermore, their heterogeneity prevents comparisons in terms of methods used, results and variables that may have influenced interpretation. For all these reasons, interpreting the impact of the VL remains challenging, although data from current and previous literature are substantially more in favor of a relationship between higher VL levels and vaccine intention or acceptance, and allow considerations on the association of VL with other variables, its mediating role and prospects for future research. In particular, we proposed a path for developing new tools to assess VL, based on selected and previous publications, along with their different methodological approaches, and sustained by a post-hoc analysis of our previous data sets.
4.2. The HL and VL Mediating Role
The mediating role of HL has been assessed in various relationships such as between education and health outcomes [
57], socio-economic and health status [
58], and, more recently, in the pandemic context, between health care system distrust and VH [
59], and between personal antecedents and vaccine confidence [
60]. Other studies have investigated effects of HL and VL when mediated by other variables, although with different objectives and methods (
Figure 3):
Using a 11-items tool derived from HLS-EU-Q47, including general questions about immunization (like “
Understand why you need vaccinations”), Jiang et al [
61] showed that the relationship between perceived HL and Covid-19 vaccine acceptance was completely mediated by attitudes toward general vaccination and self-efficacy of Covid-19 vaccine. The path from determinants to vaccine acceptance has been illustrated by Hurstak et al [
60,
62]. Authors showed in a selected population of urban adults, using a functional HL tool (Touchscreen Technology - LiTT) and an adapted vaccine confidence scale, that HL mediated the relationship between some demographic variables and vaccine confidence, which in turn mediated the relationship between HL and Covid-19 vaccine acceptance. According to Lu et al [
23], using Covid-19-VLS in the Chinese general population, all the ‘3Cs’ psychological antecedents played a significant role in mediating VL with VH, accounting for 66% and 95% of the total effect of FUVL and ICVL, respectively.
Using the same tool to check mediating effects on our previous data series [
15] (unpublished data) we found that VL (in particular ICVL) significantly mediated in the Italian general population the relationship between demographic variables, such as education level, and positive beliefs about vaccination, assumed as confidence, complacency and convenience. In their turn, beliefs partially, but significantly mediated the relationship between VL (particularly ICVL) and flu vaccine uptake.
Using a VL single item nominal tool Shon et al [
36] demonstrated the mediating effects of health beliefs (Health Belief Model’s perceived benefits, severity, and susceptibility) between flu VL and flu vaccine acceptance in students, although the literacy of influenza vaccines also directly improved the vaccination behavior of individuals without any health-mediating effects of belief. Finally, Collini et al found that vaccine confidence (assessed by the same VCI scale used by Hurstak) completely mediated the relationship between ICVL (assessed through HLVa) and the intention of nursing home personnel to get vaccinated against flu [
28].
Comparing different mediation models is challenging and should be interpreted carefully. However, these data show that VL can significantly influence vaccine acceptance both directly and indirectly. Clearly, the HL and VL roles may vary depending on the tools used. The last above Authors had previously shown that there was no significant correlation of vaccine uptake with HL when evaluated by a general functional tool (Imeter), and the VCI score and the HL score were only weakly correlated [
63]. This was confirmed in another survey using the same functional tool, where an association was found between vaccine intention and the VCI score, but not with the HL score, and the relation between vaccine intention and the HL score was not statistically significant [
64]. The same, HL did not affect the likelihood of influenza vaccination uptake and did not mediate the relationship of any independent socio-demographic variable with flu vaccination in a population of high-risk individuals, using the 6-items European Health Literacy Survey Questionnaire (HLS-EU-Q6) not including specific items on immunization [
55,
65].
In summary, available data indicates that specific VL assessment - especially ICVL - instead of general HL can better contribute to analyzing VH predictors. Indeed, in addition to its conceptual definition, VL can be explained by its ‘location', as it sits at the intersection between antecedents (moderators) and intermediate variables (mediators), partially overlapping these last (
Figure 2). The greater the involvement of VL tools in the mediation area, especially concerning beliefs, attitudes, motivations, and self-abilities, the more significant impact VL can have as a mediator and driver to promoting health behaviors and outcomes, such as vaccine acceptance, i.e. the degree to which individuals accept or refuse vaccination [
66], and vaccine uptake, i.e. the proportion of individuals who have actually received a vaccine.
4.3. Current VL Tools
The VL tools developed so far are intended for the assessment of the personal VL of the adult general population, although some have been adapted to selected adult populations.
HL and VL have received growing attention through research during the Covid-19 outbreak, where new measures have been proposed assessing coronavirus literacy and exploring attitudes and behaviors. Several tools were developed or adapted to assess HL levels from the onset of the pandemic, such as CoV-eHEALS [
67], HLS-Covid-Q22 [
68], and an updated version of the European Health Literacy Questionnaire (HLS19-Q47) [
69] including four vaccination-related questions to assess vaccine HL. In other surveys, HL was measured using the 12-item short version of HLS-EU-Q, integrated with three vaccine-related items [
70]. As mentioned, following the first tools for the assessment of VL (HLVa), specific instruments were developed to conduct studies during the coronavirus outbreak, to accurately assess VL levels, such as the Covid-19-VLS, including functional and interactive-critical subscales, in addition to measures for evaluating other variables, such as beliefs, attitudes and behaviors toward Covid-19 vaccines, despite in this instrument the evaluation of the different variables is calculated by different rating methods.
Functional literacy refers to the use of semantic and cognitive abilities like reading, writing, and knowledge of medical terms, and mathematics [
71]. The quantification of these abilities is possible using performance-based tools. Inversely, self-report measures typically evaluate the psychological aspects that underlie components like motivation, beliefs, attitudes, and the ability to engage with information and make decisions (listening, speaking, interpreting). Standardized questions are used in objective assessment to measure underlying characteristics, while subjective measurement involves, typically on Likert scales, people self-reporting to questions on their experiences about health, although it is challenging to establish a connection between persons’ responses and their actual skill [
72].
Performance-based tools appear more suitable for estimating individuals’ skills in the health care domain, while self-reported measures are better for assessing individuals’ attitudes and knowledge beyond reading and numeracy, such as understanding the value of vaccination. Considering the domains relevant to VL (disease prevention and health promotion [
73]), the use of subjective VL tools, such as HLVa, and derivative measures, seem appropriate for that scope, as they entail items related to motivation and competencies. Their construct has been validated in the general population of different regions. PCA, and Exploratory Factor Analysis (EFA), and CFA, were used to extract the latent factors defining their construct. Through parallel analysis, all have identified two separate components underlying the FUVL and ICVL items, explaining high and comparable percentages of the total variance, and significantly similar factor loadings [
56].
It has been pointed out that some latent factors might be underestimated in current VL tools [
5]. However. similar to HL, VL is a latent construct by definition [
74]: reliable measurement scales can be constructed and validated to capture VL effectively, although the results may represent aspects of VL without claiming to encompass its entirety. Indeed, we believe that despite the limitations of the current VL tools, and even if the surveys conducted so far are mostly cross-sectional and carried out primarily in the context of the pandemic, the accumulated experience remains important. The Covid-19 pandemic has likely influenced public sentiment towards the prevention of viral diseases, leading to long-term impacts on the way the general population perceives all communicable diseases. The pandemic experience will in any case affect VL with regard to other vaccines, at least in the near future. Thus, although the experience of VL tools used mainly during the Covid-19 outbreak can be considered limited, it provides a relevant reference for future research [
58].
4.4. Future Perspectives
As for HL [
74], new methods for VL measurement should explicitly refer to the domains outlined in relevant conceptual frameworks. In addition, to address the limitations of current measures, and align them as much as possible with the most recent definitions of VL, the construct of new instruments should contain and integrate items related to all the three components (motivation, knowledge, competencies), thus reducing the risk of underestimating latent factors. We attempted to determine an effective approach for incorporating them along with the three VL levels (functional, interactive, and critical) within the Health Literacy Skills Framework (HLSF), as illustrated in
Figure 2. The framework, proposed by Squiers [
20] was used for our analysis together with the Paasche-Orlow and Wolf model [
21]. We considered moderators (proximal and distal determinants), and possible mediators (i.e. variables that explain the reasons and mechanisms behind outcomes). These variables include communication, knowledge, beliefs, attitudes, behaviors, self-efficacy and competencies, incorporating concepts from various psychological theories, such as, among others, the Health Belief Model (HBM) [
36], and the Protection Motivation Theory (PMT) [
52]. We referred to tools designed for assessing these variables, relevant in the vaccination field, also taking into account the mentioned VH models (
Table 2). Based on this, we have endeavored to outline potential new VL tools.
Motivation: there are different definitions for motivation [
75]; according to the American Psychological Association) [
76], motivation refers to “a person’s willingness to exert physical or mental effort in pursuit of a goal or outcome”. It is a dynamic concept, resulting of internal and external inputs that lead to decisions and behaviors [
75]. Motivation encompasses attitudes, which includes beliefs, emotions, and evaluations. In clinical settings there are various tools for measuring motivation [
75]. Considering the consistency of the results reported so far from studies using HLVa and derived measures (Covid-19-VLS), the VL scales entailed in these tools appear suitable for assessing motivation also for future research, in particular in the interactive-critical subscale. Redundancy of questions identified during the validation process was addressed through factor analysis [
77] reducing the number of questions from 14 to 12 overall (four for FUVL and eight ICVL) [
15]. As reported above, additional analyses can be performed to further reduce the number of questions to lessen as much as possible their total number and facilitate their filling in. It is also possible to design VL tools to assess motivation by adapting items from the PMT, which explains how people respond to fear-evoking or threatening messages, or from the HBM, which explains and predicts health behaviors by examining the attitudes and beliefs, as previously demonstrated by others [
78]. The mediating role of beliefs, evaluated by the HBM, in the relationship between VL and flu vaccination has been studied in one of the selected publications [
36]. Noteworthy, the papers on behavorial change models cited in
Table 2 represent just examples of a very extensive literature.
Knowledge: is the state of being familiar with something or aware of its existence, usually resulting from experience or study (i.e. information learned) [
76] . It is part of the conceptual VL definition and a key mediating component in the VL framework. Procedural knowledge, which involves understanding how to carry out specific tasks or actions, serves as the basis for ICVL skills and fluid cognitive abilities, whereas crystallized abilities, including generalized knowledge and vocabulary, are typically more associated with functional skills, particularly in the elderly population [
71]. Understanding the relation of VL to basic cognitive abilities is important since research has shown that both general intellectual abilities and literacy are related to health [
79]. While IQ tests can provide a broad assessment of cognitive abilities, adding psychological assessments to the tools designed for the general population can be challenging. Evaluating the level of crystallized knowledge through a vaccine quiz can offer a straightforward, relevant, and efficient performance-based method. This approach has been previously employed for validating the theoretical construct of HLVa-IT, where the quiz was administered together with the VL questionnaire [
77]. In addition, vaccines quizzes can be structured to objectively evaluate the individual’s functional reading and understanding skills, when administered through face-to-face interviews or by paper-and-pencil. Regarding online surveys, there is an inherent risk that individuals may seek assistance or refer to external sources, which can potentially inflate their scores on assessments. However, measures can be taken to reduce this risk, such as underlining the anonymity of the survey, or by designing the questionnaire in a linear, one-directional flow where respondents can only move forward, and by including control or confirmation questions in the final page.
The content of the scales can be adapted over time according to the vaccines that will become available in the future, although addressing the general knowledge about vaccination and on the most common routine vaccines (D, T, Polio, Influenza) would reduce disparities. Items can be selected from vaccine scales available in the literature [
80], and online resources provided by academic [
81] and international institutions [
82]. Common questions on the knowledge of vaccines and diseases should be identified and used for comparability purposes, while taking into account cultural and socio-economic differences between populations.
Table 2.
Psychological theories explaining behaviors toward vaccination and related assessment measures, usually rated through odd-points scales. Different examples of items reported in the literature (for influenza, Covid-19, and HPV) are proposed for each component of the models, to be possibly used or adapted for building new VL tools to explore motivation and competencies, while knowledge can be assessed by performance-based measures, administering standardized stimuli.
Table 2.
Psychological theories explaining behaviors toward vaccination and related assessment measures, usually rated through odd-points scales. Different examples of items reported in the literature (for influenza, Covid-19, and HPV) are proposed for each component of the models, to be possibly used or adapted for building new VL tools to explore motivation and competencies, while knowledge can be assessed by performance-based measures, administering standardized stimuli.
Competencies: can be viewed as a set of knowledge, skills, capabilities (abilities), and behaviors that contribute to the individual’s performance [
87]. Given that knowledge on vaccines can be evaluated as described above, and that skills and abilities can be assessed through the existing VL scales (such as the interactive-critical sub-scale of HLVa), it is suggested to complete the assessment of competencies by incorporating standardized items that evaluate vaccination behavior (such as vaccines received), together with the educational level of respondents. This suggestion is based on the general understanding that individuals with higher education levels are more likely to adopt healthy behaviors [
88]. Education plays a crucial role in providing individuals with knowledge and skills required to develop competencies in various domains such as health, although HL is not solely dependent on educational levels, and these competencies can also be developed through other means in addition to education. Selected items from self-efficacy scales [
53] specifically developed and adapted to the vaccination field may be useful in completing the assessment of competencies. However, it's important to underline that competencies refer to the actual knowledge and skills possessed by an individual, while self-efficacy relates to an individual's belief in their capability to use their competencies effectively [
76].
In the literature, composite instruments are reported combining scales to evaluate different variables related to vaccination [
10,
89], while creating multidimensional composite VL tools is challenging because of the complexity of influencing factors, and the many concepts involved. Elements related to psychological content, such as motivation, beliefs, and attitudes, are often included in a variety of other tools usually administered together with HL or VL questionnaires in conducting investigations. Incorporating some of these elements in a single VL tool would standardize responses, making comparisons easier. Our proposed approach involves selecting a few items for each VL dimension and incorporating additional questions related to beliefs, attitudes, and behaviors, in addition to knowledge. The selection of these elements can be guided by meaningful grouping [
90], reliability and factor analyses, in addition to specific statistical methods to handle categorical variables, like the two-stage path analysis [
91]. These methods can ensure the construct validity of composite instruments, despite the reduction in the number of elements [
73].
Indeed, Covid-19-VLS, frequently utilized in recent studies [
5,
13,
56], can already be considered a composite VL measure. In addition to the VL subscales, it includes questions related to beliefs and attitudes underlying confidence, complacency, and convenience, in addition to coronavirus vaccines intention, and behaviors such as flu vaccine uptake (
Appendix A). Although a reduction in VL items was made compared to HLVa, we were able to further reduce their dimensionality by applying PCA as a method of data reduction. This exercise was done also by considering the results of the mediation analysis, balancing the weight of the single VL items, with the purpose to ensure the use of similar elements in future research. This involves including additional questions on knowledge and skills to new assessment scales, maintaining an acceptable total number of items in order to balance the length of the questionnaire with people's willingness to participate in the surveys. However, the decision on how many items to use should consider the trade-off between maintaining significant factors, reliability, and data interpretation. This approach will also be useful in addressing a limitation of the current VL instruments, related to a possible underestimation of their specific dimensions [
6]. In fact, the current VL instruments are derived from tools that were originally developed to assess HL in other areas of medicine (chronic patients), although they have been validated in various languages and cultural settings [
14]
In summary, using similar metrics on all elements studied, at the same time balancing them, and considering their association with each other (i.e. VL linked to beliefs and attitudes, linked to outcomes), a composite VL score can be sought for future assessments. In addition to measuring separately each scale included in a multidimensional framework - useful to observe the correlations and the mediating role of the variables between them -, adding a standardized combined index would allow simplified representation and easier interpretation of results, as well as improve statistical power [
92]. Moreover, including self-reported and performance-based elements in the same tool would facilitate a better assessment of the overall individual VL levels, without the need for additional tests. It has been shown that the combination of the results obtained using at the same time performance-based measures of functional HL and self-assessed measures of general HL may results in an increase in sensitivity (i.e. the identification of people with low HL skills) and improve the understanding of the relationship between HL and its antecedents [
93]. It is reasonable to assume that the same can be valid for VL.
4.5. Specific VL Measures
Questions included in new VL tools should be adapted to the specific medical specialty or field for which the measure will be intended. As opposed to HL where there is a huge proliferation of measures [
72], the number of tools for VL is limited. Therefore, for HL tools developed for several specific contexts and populations outside of pandemic emergencies, ideally a similar approach should be taken for VL. In addition, it will be important to develop dedicated tools for evaluating VL in different medical areas, such as routine vaccination for children, for patients and other specific categories (e.g., healthcare workers and travelers), as well as for unique psycho-physiological situations, for instance pregnancy, although existing VL tools have been used in some of them [
44,
94,
95,
96,
97]. Moreover, in the future, vaccine applications will extend beyond the prevention of infectious disease. For instance, mRNA and siRNA techniques hold potential in various healthcare areas, including oncology and diseases with genetic components [
98]. Therefore, the development of future VL instruments will extend beyond the specific area of communicable diseases.
Adolescents are another critical area for future research. The pandemic has had many negative effects on teenagers, especially in low- and middle-income countries, and vaccine coverage rates against SARS-CoV-2 in younger age groups were insufficient even in developed realms [
99]. The controversial nature of coronavirus vaccination has exacerbated the pressure on parents who make decisions about their sons’ immunization. Yet, in recommending vaccines, it is important to consider not only parents’ attitudes to increase uptake, but also adolescents’ awareness of the infective risks and their knowledge of self-consent rules, and the relevance of taking part in vaccination decisions [
100]. These aspects should be worth exploring through the development of specific VL measures. Over 40 tools are available for HL assessment in adolescents aged 10 to 7 years [
101], but none for VL, so far.
The available VL scales have shown good consistency in results across different countries, which has been proven not only comparing the average scores observed between populations, but also comparing factor analyses data [
14]. However, the cut off values have been set only arbitrarily, so far: VL has been defined as limited when the HLVa and Covid-19-VLS score is ≤2.5, or when belonging to the lower tertile of the average values observed in a given population. For example, in the mentioned 2020 survey the upper tertile bound corresponded to a score ≤2.50 for FUVL and ≤3.13 for ICVL, which allowed to define low-literate people, as already done by others [
96,
102]. While defining a universally applicable cut-off value, it would facilitate comparisons, using the lower tertile approach would more rigorously evaluate literacy levels according to local settings. In such a case, turning the score in a standardized one (observed value – mean of the sample / SD) could allow for meaningful comparisons.
Validation of future VL tools should ideally be performed internationally, to get results at the same time in more than one country, aiming to define a universally applicable threshold value, although also referring to local average scores to identify limited VL will remain. Future tools should be also proposed in prospective cohort and longitudinal studies for better understanding the causal relationship between VL and VH, and the relevance of the mediating role of VL. To improve homogeneity and comparability of populations new tools should be administered via the web in a standard manner trying to reduce as much as possible biases related to online surveys – like the social desirability bias -, using multi-item scales and combining self-report measures with other data sources, such as behavioral observations, as well as underlining anonymity. As mentioned, adding objective measures, such as about vaccine knowledge will be also helpful.
Finally, while a definition for organizational VL has been proposed [
6] and it is already mentioned in literature [
103], it is important to develop specific measures. Improving organizational VL is crucial for increasing vaccination rates, as healthcare organizations play a vital role in providing trustworthy and accessible information to the community. Specific instruments to evaluate organizational HL capacities have already been created and applied [
104,
105], It should be done the same for organizational VL, which should be the subject of dedicated research.
4.6. Limitations of This Review
Despite the use of various databases and attempts to be as comprehensive as possible, this review may not have identified all relevant recent articles in the literature, as the overall search strategy may have been biased toward public health. Searches of other databases may have resulted in other relevant publications. Furthermore, the search was conducted using only English terms, which possibly could have led to missing some studies. Limitations and the heterogeneity of online cross-sectional studies in terms of methods used, and reporting of results may have affected the interpretation of the data. Furthermore, due to heterogeneity of the results reported in the selected studies, the findings were addressed only descriptively.
Despite differences, most of the reviewed studies used the same scales validated in different populations and languages, which is a strength, considering the wide variety in rating scales of online questionnaires. Therefore, through associating various studies, we believe it has been possible to obtain a fairly accurate understanding of the current utilization of tools and the assessment of VL skills. However, this understanding remains largely descriptive, showcasing diverse values across different regions and populations. These variations are likely linked to methodological and/or local cultural differences.
Finally, in this review a post-hoc analysis of previous data sets and subsequent proposals for the development of future tools has been carried out, not to revise the results already published, which remain valid, but in view of the development of future tools, which purely reflect the point of view of the Authors and their experiences within VL domain. Comments and proposals from other research groups, hopefully numerous, will be welcome to broaden the discussion and progress on this important public health topic.