Prerpints.org logo
Preprint
Article

Developing the Social Ecology of Occupational Zoonoses Instrument: A Comprehensive Tool for Measuring Social and Behavioral Factors in Agricultural Settings

Altmetrics

Downloads

95

Views

24

Comments

0

A peer-reviewed article of this preprint also exists.

supplementary.docx (20.99KB )

Submitted:

11 July 2023

Posted:

11 July 2023

You are already at the latest version

Alerts
Abstract
This article presents the development and validation of a new instrument measuring social, cultural, and behavioral factors influencing exposure to occupational zoonoses in agricultural settings. The Social Ecological Model (SEM) and the Standards for Educational and Psychological Testing were used as guiding frameworks to ensure the instrument's validity. The instrument's content was compiled by combining the results of a scoping literature review and an expert qualitative study. The instrument items were drafted, organized, and underwent a meticulous process of revision and adjustment. It was translated into Spanish and tested in one-to-one cognitive interviews with five volunteer agricultural workers. Exploratory factor analysis was conducted for construct discrimination, and bivariate regression analyses were conducted to explore the association with exposure indicators. Evidence of validity was obtained from four out of five sources of validity evidence according to the AERA/APA’s Standards of Educational and Psychological Testing. The Social Ecology of Occupational Zoonoses (SEOZ) was successfully used to determine social and behavioral factors associated with a higher risk of exposure to occupational zoonoses. Further use of the SEOZ can provide valuable insights into developing effective interventions to improve the health and well-being of agricultural workers.
Keywords: 
Subject: Public Health and Healthcare  -   Public, Environmental and Occupational Health

1. Introduction

Agricultural workers face numerous zoonotic threats, making them highly susceptible to infections [1]. Therefore, it is crucial to understand the social and behavioral factors that influence their preventive behavior and inform the development or improvement of interventions to reduce exposure risks [2]. Among the most high-risk working environments for zoonotic exposure are dairy cattle operations, with individuals working or residing on farms, farm visitors, service providers, and veterinarians being particularly vulnerable [3,4,5]. Pathogenic agents found on dairy farms can cause diseases in farmers, farm workers, service providers, and consumers of dairy products. Salmonella [6,7], E. coli (O157:H7) [4,8,9], Campylobacter jejuni [10,11,12], and Cryptosporidium parvum [13] are the most common pathogens found. It is, therefore, crucial to address zoonotic threats and develop effective preventive measures to safeguard the health and well-being of agricultural workers.
It has been demonstrated that the behavior of the at-risk person can affect their exposure to infectious agents [14,15]. Preventing zoonotic diseases in animal-human interfaces can be challenging, as it is influenced by the social and cultural systems that drive preventive behavior [16]. Understanding the social and behavioral factors involved is necessary to develop effective interventions to reduce occupational zoonoses' impact. The Social Ecological Model is a theoretical and practical framework composed of hierarchically organized levels that comprehensively provide all the potential factors that could affect the preventive behavior of a person [17,18]. These levels are intrapersonal, interpersonal, organizational, community, and enabling. This model provides a useful framework for better understanding the multiple factors that impact preventive behaviors on farms.
To our knowledge, no tools are available to measure the social-ecological factors influencing the prevention of infectious diseases in agricultural workers. Developing new instruments is an essential part of epidemiological research, and the validity of these instruments is critical for interpreting research results and the overall validity of the entire research project.
The Standards for Educational and Psychological Testing is considered the best practice in psychometrics and provides guidelines for developing and evaluating psychometric instruments [19]. Validity is defined not as an absolute characteristic but as a continuum of evidence that can be accumulated from various sources. There are five sources of valid evidence: content, response processes, internal structure, relationship with other variables, and consequences of applying the instrument. Researchers need to pay significant attention to instrument validity to ensure the overall validity of the entire research project. The Standards provide a comprehensive framework to guide researchers through developing and validating psychometric instruments to ensure that research results are valid and meaningful [19,20].
This study aimed to develop and demonstrate evidence of measurement validity of an instrument measuring social, cultural, and behavioral factors influencing exposure to occupational zoonoses in agricultural settings. For this study, we used the Standards for Educational and Psychological Testing as a guidance framework and described the process of developing a new instrument for measuring social-ecological factors related to the prevention of occupational zoonotic diseases at the human-animal interface. We also demonstrate and discuss its diverse sources of evidence of validity.

2. Materials and Methods

A psychometric instrument, the Social Ecology of Occupational Zoonoses (SEOZ), was developed as a part of a larger research project to measure personal, interpersonal, and organizational factors affecting exposure to occupational zoonoses in agricultural settings [21]. The instrument was based on three principles: (1) appropriateness, (2) quality and clarity, and (3) validity. To assure these, a stepwise development process was conducted, including content assessment studies, critical revisions and translation procedures, and evaluations of four sources of measurement validity evidence.

2.1. Content of the instrument

The instrument's content was determined in a deductive manner guided by the Social Ecological Model (SEM) [17,18]. This model comprises factors that describe the ability to affect or change preventive behavior (Figure 1). The instrument’s content only included intrapersonal, interpersonal, and organizational measures. This decision was based on the ease of access to these levels for a potential farm-based prevention program.
Two approaches were conducted independently to assess the instrument's content: i) a scoping literature review and ii) a qualitative study with experts.

2.1.1. Scoping literature review.

The study aimed to identify published studies using the SEM to prevent infectious diseases. For this purpose, the following search string was used: (“social-ecological” OR “socio-ecological”) AND (model OR framework) AND (prevention OR intervention) AND ("infectious diseases" OR "communicable diseases" OR "transmissible diseases," this string was translated and applied to the following repositories: Science Direct, PubMed, EBSCO (Medline and Academic search premiere), and Web of Science core collection. An initial manual selection of articles was conducted based on title and abstract. Then, the selected articles underwent full-text review and data extraction for further analysis.

2.1.2. Experts qualitative study.

A crowdsourcing participatory qualitative study with subject matter experts was conducted to assess specific SEM components relevant to preventing occupational zoonosis in animal production settings. The participant inclusion criteria were university veterinarians, epidemiologists, psychologists, and occupational health specialists involved in preventing occupational zoonoses. The experts were also selected based on their willingness to participate, availability for participation, effective communication skills, experience (i.e., > three years of experience in the field), or expertise (i.e., > one article published in the last two years). A snowball sampling strategy was applied.
Expert participants were asked to respond to open-ended questions seeking to elucidate and gain insight into the potential components of the SEM relevant to the prevention of zoonotic diseases in farms. The responses were analyzed using framework-guided thematic analysis [22]. Themes and subthemes were then aligned to the SEM levels.

2.2. Item creation and review

Based on the findings of the previous content studies, a comprehensive scoping literature search was performed to identify relevant previously validated instruments containing constructs or items. Original items were drafted from scratch in English for content not identified in the literature review. The items were reviewed using Fowler Jr (2013) [22] recommendations, i.e., double-barreled questions, low clarity, ambiguousness, use of jargon, and unbalanced or biased questions. Additionally, the items underwent a thorough quality evaluation on correct spelling, wording, structure, and grammar. The questionnaire was initially organized according to the levels of the SEM. At this stage, the items were also presented for review to four volunteer graduate students from the Departments of Animal Sciences and Clinical Sciences of Colorado State University (CSU). Suggestions, questions, comments, or concerns were recorded and later discussed and adjusted (e.g., rewording, dropping, merging, splitting, etc.).

2.3. Translation process

The target population demographic is mostly Spanish-speaking laborers; thus, two native-speaker translators independently translated the questionnaire into Spanish. The two were bilingual (native Spanish) with veterinary and epidemiology backgrounds (EC & JP). The two versions were contrasted, and differences were discussed and conciliated. The translators paid attention to common local cultural jargon used by the farm workers. The Spanish version was presented to a third bilingual expert with field experience (NR) for fine adjustments.

2.4. Cognitive interviews

After approval from the CSU–Institutional Review Board (CSU-IRB), in-person cognitive interviews were conducted with five volunteer workers from a nearby dairy farm. Briefly, the participants were informed of the purpose of the interview and were asked to provide their perception of the questionnaire's relevance, intelligibility, and overall clarity. Data were collected in field notebooks and used for item adjustment. The order of items and the form of the instrument was also revised.

2.5. Data collection and participants

Once all the comments were addressed and adjustments were made, the instrument was administered to volunteer workers from dairy farms in northern Colorado. A short verbal introduction and informed consent letter were presented to the potential participants. Modest monetary compensation was included to cover their time and extra transportation fees. Workers in milking parlors, calf rearing, maternity, and hospital areas were selected due to the reported high risk for exposure to zoonotic diseases in these areas. The instrument was administered in a one-to-one interview by a trained researcher (JP) following Hartge and Cahill's [24] recommendations. Alongside, as part of the general project activities, microbiological samples were collected from the workers’ protective clothing and analyzed in the laboratory to confirm the presence of zoonotic agents. A detailed description of this process can be found in our earlier publication, Palomares et al. [25].

2.6. Statistical analysis

Variables and observations with a substantial proportion of missing responses (>20%) were removed before data analysis. Minimum residual factoring Exploratory Factor Analysis (EFA) with varimax rotation and median imputation was performed. Parallel analysis was used to select the number of factors to retain. A loading greater than |0.5| was the cut point for allocating items into factors and for interpretation. The resulting factor scores were also used as dependent variables in regression analysis to explore the statistical association with laboratory results. All the statistical tests were executed on R statistical software [26].

3. Results

3.1. Evidence of validity based on contents

3.1.1. Literature review

After full-text evaluation, 19 manuscripts were found to show compelling evidence of the use of SEM in the prevention of infectious diseases (Figure 2). Most of the articles found focused on preventing HIV transmission (9/19). Three (out of 19) articles were about risky sexual health behavior and other STDs. Malaria, dengue, hepatitis C, and tuberculosis prevention were addressed in one paper each. Two papers addressed zoonotic diseases, and one addressed the possible relationships between social behavior and infectious agents in general. From this step, 16 content items aligned with the SEM.

3.1.2. Qualitative study.

Eighteen experts were identified and reached through email. Eight of them responded positively (44% response rate). Participant experts were two university veterinary hospital faculty, one dairy industry consultant, one extension specialist, two epidemiologists and public health experts, and two occupational safety/industrial hygiene experts. Using the framework-guided thematic analysis, the initial coding yielded 58 codes. These codes were arranged into five content items that aligned with the SEM levels.
The 22 content items (16 from the literature review and 5 from experts) were used as the source of content for the instrument (Figure 3).

3.2. Evidence of validity based on response processes

We included 8 items relevant to the previously established content from the comprehensive literature review on validated instruments. These included: Risk perceptions, self-efficacy, competing demands, norms and culture, work climate and culture, organizational responsibility, and organizational support. Additionally, 143 questions were composed using the recommendations of Fowler Jr (2013) [23].
In the review process with the coauthors and the volunteer graduate students, items were adjusted, combined, or removed. One hundred and twenty-five items remained after the initial evaluation and were organized into sections corresponding with the levels of the SEM. Also, sections on demographics and preventive practices measures were included per the greater research project objectives. From the cognitive interviews, seven additional items that induced biased or inclined to falsify responses or that seemed difficult for the responders were dropped.
A total of 54 workers were reached and asked to participate. Of those, 42 workers volunteered to respond to the SEOZ instrument (78% response rate). The main reasons for non-participation were time restrictions or lack of transportation availability after the interview. On average, the instrument was completed in 35-40 minutes. The filled database had approximately 4% of missing values. Sixteen items and one responder were dropped due to their high proportion of missing responses (>20%); this procedure reduced the missing values to 1.65%.

3.3. Evidence of validity based on the internal structure

According to the parallel analysis, ten factors had equal or larger eigenvalues than those randomly simulated. Therefore, they were retained (Figure 4). Results from the EFA (Exploratory Factor Analysis) are summarized in Table S1. The retained ten factors accounted for 71% of the total variance (eigenvalues ranging from 9.13 to 5.15). Of the 102 items, 74 (73%) were relevant contributors to the component's variance.
Clear construct discrimination was identified for most of the factors. The first factor (8.95% of total variance) contains an organizational construct related to management perception and sense of belonging. The second factor (8.63 % of total variance) contains an intrapersonal construct related to knowledge and risk perception. The self-efficacy construct was the third factor (8.02% of total variance). The fourth factor (7.38% of total variance) contained the construct of perception toward peers and supervisors. The fifth factor (7.37% of total variance) is a construct of attitudes towards reporting and seeking health care due to injuries.
Similarly, the sixth factor (6.89% of total variance) is a construct of attitudes towards reporting and seeking health care when aware of having symptoms of infection. Job satisfaction was compiled in the seventh factor (6.67% of total variance). Safety culture and climate were compiled in the eighth factor (6.2% of total variance). And organizational responsibility and job control were constructs discriminated in the 9th and 10th factors, respectively (5.79% and 5.04% of total variance).

3.4. Evidence of validity based on relationships with other variables

This evidence was assessed a posteriori by regression analysis with factors as dependent variables and laboratory results from workers' samples as dependent variables [25]. A significant association was found between four factors and exposure to potentially dangerous zoonotic agents. Self-efficacy and negative workplace perceptions were risk factors of exposure (OR: 1.43, 95% CI 1.11-2.22; and 1.22, 95% CI 1.02-1.53, respectively). In contrast, Knowledge, risk perception, and good perceptions of supervisors and coworkers were protective factors (OR: 0.91, 95%CI 0.82-0.99; and 0.89, 95%CI 0.79-0.98, respectively). Please refer to our previous publication [25] for a more detailed interpretation of these results.

4. Discussion

We have developed an instrument for measuring social, cultural, and behavioral factors influencing the prevention of zoonoses in animal production occupational settings. We effectively used the Social Ecological Model as a guiding framework to realize the instrument's content, and we gathered evidence of measurement validity from various sources according to the Standards for Educational and Psychological Testing guidelines. This development's purpose was to assure the quality and accuracy of data collection to inform the development or improvement of prevention interventions.
We used two complementary sources of content: a scoping review of the literature and a qualitative elicitation of factors from experts. Both literature reviews and expert assessment have been broadly used to assess content, but to the extent of our knowledge, they have not been used together to assess social-ecological variables [27,28]. Finding convergence and complementarity between both approaches is evidence that the content items are relevant and representative of the constructs that are intended to be measured [27,29].
Clarity and quality were effectively achieved through extensive review steps and cognitive interviews. These methods have been recommended as the preferred method for assessing response processes [30].
Assessing the instrument's dimensions with exploratory factor analysis provided evidence of validity based on internal structure [31]. While most components showed evident discrimination of constructs, we found individual items that did not belong to the construct. However, these few items had lower loadings <|0.6| than others, more representative of the constructs.
Finding some factors associated with exposure to zoonotic diseases is evidence of validity based on relationships with other variables. Self-efficacy, knowledge perception, peer and supervision perceptions, and management perceptions have been previously identified as drivers of occupational preventive behavior [32,33,34]. These findings provide grounds to develop or improve prevention programs to reduce occupational risks in similar settings.
Evidence of validity based on the consequences of applying the instrument refers to the direct or indirect effects caused by the application of an instrument [35]. In this study, we did not evaluate whether the instrument's application negatively affected the workers. However, following the ethical recommendations, we ensured that participation in the study did not cause any harm or negative consequences to the participants. We ensured that survey responses could not be traced back to its respondents to avoid any potential confidentiality breach. Also, we tried not to interfere with the workers' routine. All the participants were asked to participate in the study voluntarily and could opt out of participation at any time. Requesting permission from the farm's administration, performing the interviews during non-shift hours, and maintaining strict confidentiality of participants are indicators of our efforts to avoid negative repercussions.
Although the accumulative amount of validity evidence found for the SEOZ instrument is promising, the findings presented here should not be considered without considering its limitations. We did not conduct a quantitative content study for content, but a scoping review and a qualitative expert study approach were appropriate for the study's purpose and the research project's goals. The cognitive interviews were conducted in a small sample of a very homogeneous group of workers. Thus further testing may be necessary to adapt this instrument to other communities. Also, the language used, and the translation was conducted for a specific target population. The limited access to different farms and a larger sample of workers was a barrier to the study heterogeneity. As a result, we lacked the power to find any multivariable or multilevel effect. Another limitation is that the SEOZ was developed for this in a specific research project, so its reliability was not evaluated. More extensive testing is required with a larger and heterogeneous sample to overcome these limitations.

5. Conclusions

In conclusion, developing and validating a robust psychometric instrument required greater effort than perceived or anticipated. We believe the SEOZ instrument's development significantly contributes to occupational health and safety in agricultural settings. The SEOZ instrument can provide valuable insights into preventing and controlling zoonotic diseases in farm settings by measuring social, cultural, and behavioral factors. This instrument can inform the development of effective interventions to improve agricultural workers' and their communities' health and well-being.

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org.

Author Contributions

Conceptualization, J.PV., and I.RM.; methodology, J.PV., and S.R.; validation, and formal analysis, J.PV. and S.R.; investigation, J.PV., I.RM, and. M.S; resources, I.RM, and. M.S.; writing—original draft preparation, J.PV.; writing—review and editing K.S. and M.S.; supervision, M.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work was partially supported by the Center for Disease Control and Prevention - National Institute for Occupational Safety and Health (CDC-NIOSH) through the High Plains Intermountain Center for Agricultural Health & Safety (HICAHS) (grant number 5 U54OH008085-16-00). CIDEIM and the Fogarty International Center at the National Institutes of Health (D43 TW006589).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Colorado State University (protocol code 15-6168H, date of approval August 17, 2016).

Data Availability Statement

Data of instrument responses are available upon request to the corresponding author.

Acknowledgments

To Megan Dietz, MSc, for her contribution to the qualitative analyses. To Dr. Gwen Fisher and her team for their contribution to identifying previously validated instruments. To Dr. Elsa Cardenas for her contribution to translation. To the farm personnel and participants.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the study's design; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Battelli, G. Zoonoses as Occupational Diseases. Vet Ital 2008, 44, 10. [Google Scholar]
  2. Menger, L.M.; Rosecrance, J.; Stallones, L.; Roman-Muniz, I.N. A Guide to the Design of Occupational Safety and Health Training for Immigrant, Latino/a Dairy Workers. Front. Public Health 2016, 4. [Google Scholar] [CrossRef]
  3. Collins, J.D.; Wall, P.G. Food Safety and Animal Production Systems: Controlling Zoonoses at Farm Level. Rev. Sci. Tech.-Off. Int. Épizooties 2004, 23, 685–700. [Google Scholar] [CrossRef] [PubMed]
  4. Crump, J.A.; Sulka, A.C.; Langer, A.J.; Schaben, C.; Crielly, A.S.; Gage, R.; Baysinger, M.; Moll, M.; Withers, G.; Toney, D.M. An Outbreak of Escherichia Coli O157: H7 Infections among Visitors to a Dairy Farm. N. Engl. J. Med. 2002, 347, 555–560. [Google Scholar] [CrossRef]
  5. Kassenborg, H.D.; Hedberg, C.W.; Hoekstra, M.; Evans, M.C.; Chin, A.E.; Marcus, R.; Vugia, D.J.; Smith, K.; Ahuja, S.D.; Slutsker, L.; et al. Farm Visits and Undercooked Hamburgers as Major Risk Factors for Sporadic Escherichia Coli O157:H7 Infection: Data from a Case-Control Study in 5 FoodNet Sites. Clin. Infect. Dis. 2004, 38, S271–S278. [Google Scholar] [CrossRef]
  6. Rodriguez-Rivera, L.D.; Cummings, K.J.; Loneragan, G.H.; Rankin, S.C.; Hanson, D.L.; Leone, W.M.; Edrington, T.S. Salmonella Prevalence and Antimicrobial Susceptibility Among Dairy Farm Environmental Samples Collected in Texas. Foodborne Pathog. Dis. 2016, 13, 205–211. [Google Scholar] [CrossRef]
  7. Seligmann, E.; Saphra, I.; Wassermann, M.; others Salmonella Infections in Man. An Analysis, of 1,000 Cases Baeteriologically Identified by the New York Salmonella Center. Am. J. Hyg. 1943, 38, 226–249. [Google Scholar]
  8. Comstock, N.; Towle, M.; Warner, A.; Reynolds, S.; Durso, L.; Campbell, C.; Kiefer, M.; Bosch, S. Epidemiologic and Occupational Investigation of an Escherichia Coli O111 Outbreak Associated with a Correctional Facility Dairy–Colorado, 2010. Morb. Mortal. 2012, 61. [Google Scholar]
  9. Hancock, D.D.; Besser, T.E.; Kinsel, M.L.; Tarr, P.I.; Rice, D.H.; Paros, M.G. The Prevalence of Escherichia Coli O157.H7 in Dairy and Beef Cattle in Washington State. Epidemiol. Infect. 1994, 113, 199–207. [Google Scholar] [CrossRef]
  10. Besser, T.E.; LeJeune, J.T.; Rice, D.H.; Berg, J.; Stilborn, R.P.; Kaya, K.; Bae, W.; Hancock, D.D. Increasing Prevalence of Campylobacter Jejuni in Feedlot Cattle through the Feeding Period. Appl. Environ. Microbiol. 2005, 71, 5752–5758. [Google Scholar] [CrossRef]
  11. Gilpin, B.J.; Scholes, P.; Robson, B.; Savill, M.G. The Transmission of Thermotolerant Campylobacter Spp. to People Living or Working on Dairy Farms in New Zealand. Zoonoses Public Health 2008, 55, 352–360. [Google Scholar] [CrossRef]
  12. Kwan, P.S.L.; Birtles, A.; Bolton, F.J.; French, N.P.; Robinson, S.E.; Newbold, L.S.; Upton, M.; Fox, A.J. Longitudinal Study of the Molecular Epidemiology of Campylobacter Jejuni in Cattle on Dairy Farms. Appl. Environ. Microbiol. 2008, 74, 3626–3633. [Google Scholar] [CrossRef] [PubMed]
  13. Smith, R.P.; Chalmers, R.M.; Mueller-Doblies, D.; Clifton-Hadley, F.A.; Elwin, K.; Watkins, J.; Paiba, G.A.; Hadfield, S.J.; Giles, M. Investigation of Farms Linked to Human Patients with Cryptosporidiosis in England and Wales. Prev. Vet. Med. 2010, 94, 9–17. [Google Scholar] [CrossRef] [PubMed]
  14. Albarracín, D.; Gillette, J.C.; Earl, A.N.; Glasman, L.R.; Durantini, M.R.; Ho, M.-H. A Test of Major Assumptions About Behavior Change: A Comprehensive Look at the Effects of Passive and Active HIV-Prevention Interventions Since the Beginning of the Epidemic. Psychol. Bull. 2005, 131, 856–897. [Google Scholar] [CrossRef] [PubMed]
  15. Kashfi, S.M.; Khani Jeihooni, A.; Rezaeianzade, A. Effect of Health Workers’ Training Programs on Preventive Behavior of Leishmaniosis Based on BASNEF Model. J. Res. Health Sci. 2012, 12, 114–118. [Google Scholar]
  16. Donham, K.J. Zoonotic Diseases: Overview of Occupational Hazards in Agriculture. In Agricultural medicine: rural occupational and environmental health, safety, and prevention; Thelin, A., Donham, K.J., Eds.; John Wiley & Sons: Hoboken, New Jersey, 2006; pp. 437–464. ISBN 978-1-118-64720-2. [Google Scholar]
  17. McLeroy, K.R.; Bibeau, D.; Steckler, A.; Glanz, K. An Ecological Perspective on Health Promotion Programs. Health Educ. Q. 1988. [Google Scholar] [CrossRef]
  18. Stokols, D. Translating Social Ecological Theory into Guidelines for Community Health Promotion. Am. J. Health Promot. 1996, 10, 282–298. [Google Scholar] [CrossRef] [PubMed]
  19. Standards for Educational and Psychological Testing 2014 Edition; American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (U.S.), Eds.; American Educational Research Association: Washington, DC, 2014; ISBN 978-0-935302-35-6.
  20. Measurement Validity. In Research methods in applied settings: an integrated approach to design and analysis; Gliner, J.A., Morgan, G.A., Leech, N.L., Eds.; Routledge: New York, 2009 ISBN 978-0-8058-6434-2.
  21. Palomares Velosa, J.E. Social Ecological Determinants of Occupational Zoonotic Disease Exposure on Colorado Dairy Farms. Dissertation, Colorado State University. Libraries, 2019.
  22. Braun, V.; Clarke, V. Thematic Analysis. In APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological.; Cooper, H., Camic, P.M., Long, D.L., Panter, A.T., Rindskopf, D., Sher, K.J., Eds.; American Psychological Association: Washington, 2012; pp. 57–71. ISBN 978-1-4338-1005-3. [Google Scholar]
  23. Designing Questions to Be Good Measures. In Survey Research Methods; Fowler Jr, F.J., Ed.; SAGE Publications, 2013 ISBN 978-1-4833-2359-6.
  24. Hartge, P.; Cahill, J. Field Methods in Epidemiology. In Modern epidemiology; Rothman, K.J., Greenland, S., Lash, T.L., Eds.; Wolters Kluwer Health, Lippincott Williams & Wilkins: Philadelphia Baltimore New York London Buenos Aires Hong Kong Sydney Tokyo, 2008; pp. 492–510 ISBN 978-1-4511-9005-2.
  25. Palomares Velosa, J.E.; Salman, M.D.; Roman-Muniz, I.N.; Reynolds, S.; Linke, L.; Magnuson, R.; McConnel, C.S.; Rao, S. Socio-Ecological Factors of Zoonotic Diseases Exposure in Colorado Dairy Workers. J. Agromedicine 2020, 0, 1–11. [Google Scholar] [CrossRef]
  26. R Core Team R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2021.
  27. Haynes, S.N.; Richard, D.; Kubany, E.S. Content Validity in Psychological Assessment: A Functional Approach to Concepts and Methods. Psychol. Assess. 1995, 7, 238. [Google Scholar] [CrossRef]
  28. Koller, I.; Levenson, M.R.; Glück, J. What Do You Think You Are Measuring? A Mixed-Methods Procedure for Assessing the Content Validity of Test Items and Theory-Based Scaling. Front. Psychol. 2017, 8. [Google Scholar] [CrossRef]
  29. Newman, I.; Lim, J.; Pineda, F. Content Validity Using a Mixed Methods Approach: Its Application and Development Through the Use of a Table of Specifications Methodology. J. Mix. Methods Res. 2013, 7, 243–260. [Google Scholar] [CrossRef]
  30. Padilla, J.-L.; Benítez, I. Validity Evidence Based on Response Processes. Psicothema 2014, 26, 136–144. [Google Scholar] [CrossRef] [PubMed]
  31. Rios, J.; Wells, C. Validity Evidence Based on Internal Structure. Psicothema 2014, 108–116. [Google Scholar] [CrossRef]
  32. Neal, A.; Griffin, M.A. Safety Climate and Safety at Work. In The psychology of workplace safety; Barling, J., Frone, M.R., Eds.; American Psychological Association: Washington, 2004; pp. 15–34. ISBN 978-1-59147-068-7. [Google Scholar]
  33. Pickett, W.; Brison, R.J.; Niezgoda, H.; Chipman, M.L. Nonfatal Farm Injuries in Ontario: A Population-Based Survey. Accid. Anal. Prev. 1995, 27, 425–433. [Google Scholar] [CrossRef]
  34. Smith-Crowe, K.; Burke, M.J.; Landis, R.S. Organizational Climate as a Moderator of Safety Knowledge–Safety Performance Relationships. J. Organ. Behav. 2003, 24, 861–876. [Google Scholar] [CrossRef]
  35. Lane, S. Validity Evidence Based on Testing Consequences. Psicothema 2014, 127–135. [Google Scholar] [CrossRef]
Preprints 79141 g001
Figure 1. The Social-ecological model. Adapted from McLeroy et al. (1988) [17].
Figure 1. The Social-ecological model. Adapted from McLeroy et al. (1988) [17].
Preprints 79141 g001
Preprints 79141 g002
Figure 2. Flow chart for the scoping literature review process.
Figure 2. Flow chart for the scoping literature review process.
Preprints 79141 g002
Preprints 79141 g003
Figure 3. Combined content items from the scoping review and the expert survey.
Figure 3. Combined content items from the scoping review and the expert survey.
Preprints 79141 g003
Preprints 79141 g004
Figure 4. Results of parallel analysis.
Figure 4. Results of parallel analysis.
Preprints 79141 g004
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated