Introduction

Despite the paramount importance of vaccines for improving global health, vaccine hesitancy has a long history, and persists worldwide [1]. Given the deleterious effects of the COVID-19 pandemic to individuals and society, and the positive impact of COVID-19 vaccination, it is particularly crucial to understand those factors that affected COVID-19 vaccine acceptance and hesitancy among different groups including mental health professionals. A recent systematic review of 51 studies found that vaccine hesitancy and avoidance were highly prevalent, not only in the general public, but also among health care workers, emphasizing the need to further investigate obstacles to vaccine uptake, and to develop appropriate evidence-based interventions [2]. Attitudes are causally linked to health intentions and behaviour with a medium effect size [3], emphasizing the importance of measuring these cognitions and considering them for targets of interventions intended to increase vaccination rates. Therefore, the availability of a self-report measure of COVID-19 vaccine knowledge and attitudes could be useful in achieving this important public health goal.

COVID-19 negatively impacts physical and mental health, and given the important role of health care workers, including mental health professionals, in providing interventions and in vaccine advocacy, it is particularly important to understand vaccine attitudes in this group. A recent scoping review of vaccine intentions in health care workers identified 26 papers and emphasized a range of concerns about vaccine safety and efficacy [4]. Our current understanding of predictors of vaccine hesitancy in health care workers is limited, with even fewer data available about vaccine hesitancy in mental health professionals [4]. The importance of attitudes toward vaccines in this group is underscored by evidence that individuals with serious mental illness may be at heightened risk of infection and increased mortality [5]. Furthermore, protection of an already fragile and understaffed mental health workforce is a priority [6].

The World Health Organization’s Global Practice Network (GCPN) is comprised of more than 16000 mental health professionals representing a diverse range of geographic regions and settings, and provides a unique opportunity to investigate professional’s knowledge and attitudes from a global perspective. We aimed to investigate these professionals’ knowledge about and attitudes toward vaccination by administering a new 8-item measure, the COVID-19 Vaccine Attitudes Questionnaire (C-VAQ), to members of the GCPN. We investigated whether sociodemographic variables, vaccination status, as well as country mortality rate and stringency index at the time of data collection predicted C-VAQ scores.

Material & methods

The WHO’s Global Clinical Practice Network (GCPN) and the “Longitudinal COVID-19 Survey of Mental Health Professionals Project (LoCS MHP)” have been described elsewhere in more detail [7]. Briefly, the GCPN is a multilingual network of mental health clinicians, mainly psychiatrists and psychologists, representing all regions of the world [8]. To be eligible for the GCPN, members must have completed their professional training and be formally authorized to provide mental health services in their countries. At the time of the study in July 2021, the GCPN comprised more than 16000 participants from 163 countries, with excellent representation of clinicians working in low and middle-income countries. The “LoCS MHP” is an international collaboration funded by the Canadian Institute of Health Research. The “LoCS MHP” is a three-wave longitudinal internet-based survey study on the effects of the COVID-19 pandemic on numerous facets of mental health professionals’ practice and well-being (e.g., [9]). Questions related to vaccine attitudes and concerns were included in the third wave of data collection, which occurred between July 28 to September 7, 2021. After the initial invitations to participate were sent out, follow up email reminders were sent to those GCPN members who had not yet responded one week after the initial invitation and one week after the first reminder. The survey was disseminated on line using Qualtrics (Provo, UT, USA) and conducted in Chinese, English, French, Japanese, Russian, and Spanish. Survey questions were developed in English and translated by experts fluent in these languages and affiliated with the GCPN’s International Advisory Group.

GCPN members who at the time of registration had indicated that they were proficient in the corresponding study language were sent an email invitation containing an individualized survey link. Participants had previously attested to their language proficiency as part of the information provided when they registered as a member of the GCPN. Upon accessing the link, participants were asked to read a description of the study and provide their consent to participate.

Several sources of bias are inherent in questionnaire research including sampling bias, response bias, and social desirability [10]. GCPN members are from all regions of the world but may not be a representative sample. In order to ensure that the sample of GCPN members who responded to the survey were representative of the overall membership, differences in demographic variables were examined across non-respondents and respondents. To avoid the potential of response bias, the C-VAQ includes four reverse scored items. Internal consistency as well as the mean, range, and standard deviation of scores were examined to ensure that responses were non-random. Social desirability to conform with the majority view in the medical community that vaccination againt COVID-19 is beneficial, participants may have presented themselves in a more vaccine-positive light than reflects their actual beliefs. To control for such a bias, vaccination status was included as a variable in the analyses.

COVID-19 Vaccine Attitudes Questionnaire (C-VAQ)

Questionnaire development.

The COVID-19 Vaccine Attitudes Questionnaire (C-VAQ) was developed through a review of the existing literature at the time including media reports and gray literature as well as group discussions and an expert review by the LoCS MHP International Advisory Group (IAG). Item selection and inclusion was based on the common causal factors of theories of health behaviour change [3], which posit that attitudes, self-eficacy, and norms each contribute to intention to change and actual change in health behaviour. The IAG for this project comprised internationally known psychiatrists and psychologists representing all WHO global regions and many of the countries hardest hit by the pandemic (Canada: CSK, KD, P.R. China (MZ), Germany (WG), India (PS), Japan (SK), Mexico (MEM), Nigeria (OG), Russian Federation (MK), Spain (JA), and USA (KP, GMR).

Group discussions with the IAG took place virtually on three occasions and were guided by those themes identified by two of the co-authors (GMR and TJR) from their literature search. The C-VAQ addresses themes that appear in the emergency literature on COVID-19 vaccine knowledge and uptake. Themes included: 1) confidence in safety and efficacy of the vaccine, 2) beliefs about the vaccine, 3) knowledge about the vaccine, and 3) feelings about the vaccine. The IAG were tasked with creating a measure that could easily be administed. They discussed and reached consensus on 8 items (see Table 1). The C-VAQ was then reviewed by expert members of the IAG who determined that the 8 items selected met the requirements of content and face validity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Items of the COVID-19 Vaccine Attitudes Questionnaire (C-VAQ).

https://doi.org/10.1371/journal.pmen.0000018.t001

The C-VAQ was administrated to participants. The C-VAQ comprised 8 items that were self-rated on a 5-point Likert-like scale. The C-VAQ has a total score range of 0 to 40, with the last four items reverse scored such that higher scores reflect greater confidence in the COVID-19 vaccine and vaccination process. These items assess knowledge of and attitudes towards COVID-19 vaccines. Confirmation of the content validity of the C-VAQ is suggested by the partial overlap of items with a longer, 39-item questionnaire, that was developed and validated for use in the general population using a mixed methods study design in India [11].

In addition to the C-VAQ, participants were asked a single question with twelve potential options from which they could select multiple responses to solicit their concerns related to the COVID-19 vaccine or the vaccination process (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. COVID-19 concern question.

https://doi.org/10.1371/journal.pmen.0000018.t002

Predictor variables

Using a social ecological framework, factors associated with COVID-19 vaccine attitudes can be grouped into intrapersonal, interpersonal, institutional, community, and public policy factors [12]. In our survey, sociodemographic variables included age, gender, global region as defined by WHO, country income level, profession, and years of professional experience (Table 3). We also assessed whether people had been infected with COVID-19, and whether people had been vaccinated against COVID-19 with at least one dose. We summarized the number of concerns of the participants as a proxy for the degree of vaccine-related worries. Finally, we assessed organizational influences based on whether participants considered that the organization’s leadership communicated effectively to staff about vaccine availability and procedures, and made educational materials available about vaccine safety, efficacy, and side effects.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Sociodemographic characteristics of participants (N = 1,931).

https://doi.org/10.1371/journal.pmen.0000018.t003

We calculated the average new COVID-19 mortality rate by country in which GCPN members were working at the time of data collection [13]. The stringency index, which is a composite measure based on nine indicators of the strictness of each country’s response to COVID-19 (e.g., stay-at-home orders, school and workplace closures, travel bans), was also included in the analyses. Stringency was rescaled to a value from 0 to 100, where 100 was set as representative of the strictest COVID-19 public health measures to control the spread of the virus [14].

Results

Internal consistency and factor structure of the Vaccine Attitudes Questionnaire (C-VAQ)

The mean total score for the C-VAQ was found to be 31.8 (SD = 5.2, Range = 8–40) suggesting a good distribution of scores as is expected for a measure of attitudes. Cronbach’s alpha for the C-VAQ was 0.87. Internal consistency was not significantly affected if any one item was dropped. However, item level statistics suggest that the last four items should be reverse scored, which is logical given that they refer to negative attitudes about COVID-19 vaccination. Furthermore, the analysis suggested that the final item (Item 8) could be eliminated without affecting internal consistency. The polychoric correlation showed a strong to moderate correlation between items (Table 4). The scree plot analysis suggested retaining a single factor (Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Scree plot for factors of the COVID- 19 Attitudes Questionnaire (C-VAQ).

https://doi.org/10.1371/journal.pmen.0000018.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Polychoric matrix between items related to attitudes toward COVID-19 vaccine.

https://doi.org/10.1371/journal.pmen.0000018.t004

Predictor model

A multiple-linear regression model was performed to predict C-VAQ scores based on the following variables: age, gender, WHO region, country income level, profession, and years of professional experience, country mortality rate, and the stringency of each country’s response to COVID-19. The regression model revealed statistically significant results for younger age (β = -.007, CI = -.012 to -.001), living in a low- or -middle income country (β = -.324, CI = -.444 to -.203), and living in an Asian country, which all predicted lower C-VAQ scores (β = -.310, CI = -.575 to -.046) suggesting more negative attitudes toward vaccination (Table 5). In contrast, having received a COVID-19 vaccine was associated with higher C-VAQ scores (β = 1.304, CI = 1.155 to 1.452) suggesting more positive attitudes toward vaccination. The number of concerns participants endorsed about COVID-19 vaccination was associated with lower C-VAQ scores (β = -.214, CI = -.233 to -.194) suggesting more negative attitudes toward vaccination. Finally, increased country mortality rate (β = -.039, CI = .004 to .075) and higher country stringency index (β = .004, CI = .001 to .007) were associated with higher C-VAQ scores suggesting more positive attitudes toward vaccination.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Results of multiple-linear regression predicting C-VAQ scores (N = 1,931).

https://doi.org/10.1371/journal.pmen.0000018.t005

Discussion

The main findings of this study were: 1) a newly constructed measure of vaccine attitudes, the Vaccine Acceptance Quetionnaire (C-VAQ), has good internal consistency, with items loading on to a single factor, 2) having received a COVID-19 vaccine was significantly associated with higher C-VAQ scores, whereas having concerns about COVID-19 vaccination was significantly associated with lower C-VAQ scores; 3) younger age, residing in a low-and-middle income country, and living in an Asian country were all significantly associated with lower C-VAQ scores; and 4) increased COVID-19 mortality rate and higher stringency index were significantly associated with higher C-VAQ scores.

Our finding that having received a COVID-19 vaccine was significantly associated with higher C-VAQ scores provides some external validation of the measure. The measure also demonstrates some convergent validity as demonstrated by the significant negative association between C-VAQ scores and the number of concerns endorsed by participants about COVID-19 vaccination. It is also consistent with findings from experimental studies that demonstrate that changes in attitudes predict changes in health behaviours [3]. A considerably longer questionnaire assessing COVID-19 attitudes also yielded only one factor and had similar internal consistency to the C-VAQ [11]. The C-VAQ, which was translated into Chinese, English, French, Japanese, Russian, and Spanish for the present study adds to the literature a new relatively short measure that can be used by governments, organizations and researchers to efficiently measure vaccine attitudes.

Previous work has demonstrated that a range of sociodemographic factors may be associated with greater vaccine hesitancy in both health care workers and the general population [2]. It is not surprising that such sociodemographic predictors are also apparent among the attitudes of mental health clinicians toward vaccination. Nevertheless, it is important to note that there may be significant regional variation in such findings [18]. For example, within the general population, Solís Arce et al. (2021) [19] found that those living in low- and middle-income countries were more accepting of COVID-19 vaccines, particularly if guidance comes from health care workers. In the present study, we found evidence for the opposite effect such that mental health clinicians living in low- and middle-income countries as well as in Asian countries, particularly Japan, expressed more negative attitudes toward COVID-19 vaccination. This finding highlights the importance of interventions to address these beliefs in these regions. Further work must be done to understand why mental health professionals living in low- and middle-income countries, but not the general population [19], hold more negative vaccine attitudes.

It is noteworthy that higher country mortality rate and stringency index were significantly associated with higher C-VAQ scores. While environmental factors associated with vaccine attitudes may include exposure to myths about vaccines, these findings suggest that real threat from the pandemic may act to encourage positive attitudes toward vaccines. However, we cannot rule out the possibility that countries with high stringency and COVID-19 death rates had stronger programs to encourage vaccination, particularly among health professionals. A 5A model (addressing vaccine acceptability, accessibility, affordability, awareness, and activation–or nudges to use vaccine) may be useful in understanding vaccine uptake among adults [20], and in developing multi-prong interventions that address individual, organizational, and societal factors to improve such uptake [21,22].

The C-VAQ was found to possess excellent internal consistency with items loading on to a single factor. Our results suggest that the C-VAQ is a useful measure that can be used internationally to measure COVID-19 vaccine attitudes. Previous work on COVID-19 vaccine beliefs has often used a simple yes/no approach, focusing on intent to obtain vaccination [4,21]. A Likert-like measure of acceptability and hesitancy may allow for a more detailed assessment of inter-individual variations and so for more nuanced statistical analyses. A longer scale may allow assessment of different factors that contribute to vaccine attitudes, but this must be weighed against respondent burden. Future research on the C-VAQ’s sensitivity to change among international samples would be useful in the context of evaluating programs that aim to increase vaccine uptake among health care professionals.

A number of limitations deserve emphasis. First, member of the GCPN generously volunteer their time to participate in a range of research studies, and may not be systematically representative of mental health professionals. Nevertheless, the GCPN does provide access to a broad range of professionals working in diverse geographic regions and settings. Second, the design of this study does not allow determination of causality. Further work is needed to assess the precise nature of the relationships between attitudes towards COVID-19 vaccination, behavior with regards to vaccination, and social and political context. The development of the C-VAQ occurred during 2020–21 as part of an urgent call by the World Health Organization’s Global Research Roadmap [23] and funding agencies (the present study was funded by a special COVID-19 fund by the Canadian Institutes for Health Research) to address the pandemic. Furthermore, the data collection occurred at the height of the pandemic when public health measures and restrictions were in place across various countries. Variables not considered in the present report may have had a significant influence on the acceptance of vaccinations by mental health professionals at that time. While, such public measures are currently significantly less restrictive in many countries, our findings may have relevance to future pandemics. Finally, the present study examined internal consistency, factor structure, and predictive validity of the C-VAQ. Future research should validate the C-VAQ, in particular assessing convergent and divergent validity of this new measure. The availability of this measure in the published literature in advance of a future pandemic is useful. Investigating the generalizability of this brief and practical measure to other public health vaccination programs would also be beneficial. The study also has a number of notable strengths. The study includes a large sample size of GCPN members from across all five WHO regions including clinicians working in LAMICs. The multilingual nature of the study allowed for a broader participation of clinicians. Finally, the study provides the field with a brief, publicly available, free measure that can be easily administered to assess COVID-19 vaccine attitudes.

Conclusion

Employment of the C-VAQ in the World Health Organization’s CGPN was useful in demonstrating the extent to which additional work is needed to improve mental health clinician’s attitudes towards COVID-19 vaccines around the world. Lack of vaccination in this group (7.1% of professionals reported not being vaccinated) is particularly worrisome in light of evidence that individuals with serious mental illness may be heightened risk of infection and increased mortality [5]. Furthermore, protection of an already fragile mental health workforce is a priority particulary with evidence of growing prevalence of mental health conditions during the pandemic (e.g., [24]). While increased intervention efforts may be particularly pertinent for a number of sociodemographic groupings, relatively high levels of vaccine hesitancy in clinicians and communities around the world also suggest the need for broad, multi-pronged interventions that address negative attitudes among large numbers of individuals in future pandemics.

Acknowledgments

We are grateful to the GCPN members for participating in the study.