Introduction
It has been suggested that the demand for food from animal agriculture will double by 2050.
a This demand is resulting from a growing population, as well as limits placed on the availability of land, water and energy as a result of these population pressures.
1 Strategies to address these increasing demands require a strong foundation in animal, veterinary and agricultural research in order to increase productivity, food safety, and animal welfare whilst decreasing costs to consumers and negative environmental implications.
1 Furthermore, these strategies require research into changing consumer preferences associated with animal products in various locales.
There are clear evidence gaps about food systems stemming from a lack of primary data and fragmentation across systems and actors.2 These absences have led to a situation where the impact of legislation on livestock animals and the economics of farming practices can only be approximated with limited precision or accuracy due to lack of underlying data.2 However, we argue that it is not just a lack of primary data that thwarts progress: secondary sources of data have been underdeveloped and could greatly assist in identifying evidence gaps.
Syntheses of the best available evidence are critically important as enablers of translation of research into practice, whether through practical implementation or via policy reform. It is also increasingly recognised that evidence syntheses can assist in reducing wastage of declining research resources by highlighting which types of potential research are truly novel and justified based on what is already known.3 However, many of the current synthesis tools, methods, and processes are limited and not applicable to disciplines of science involving animals where outcomes go well beyond health, and considerations about external validity and applicability are often varied and inconsistent both within the animal science discipline and in comparison to traditional fields. In this paper we present an overview on the importance of incorporating robust evidence syntheses to the animal sciences and suggest priority areas for methodological development in this field. This call to action is directed towards policy makers and those conducting evidence synthesis is this field, with the hope to stimulate international collaboration and investment in methodological development.
Why Should We Care?
Today’s society demands safe, affordable, and high-quality animal products. Production should occur in sustainable and culturally sensitive ways with attention to increasing animal welfare and reducing adverse environmental impacts arising from animal agriculture.4 There is growing interest in new technological advances such as improved breeding methods in relation to production goals, automation of systems, and nutritional or therapeutic interventions to maximise productivity. There is an abundance of research evidence around these topics, as well as continued funding of this type of research by governments and industry bodies to further improve productivity, revenue, welfare, and sustainability. These topics are ripe for evidence syntheses which would help to inform both whether these new technologies and approaches should be introduced to practice and, if so, how they should be.
However, it is not unusual for goals associated with animal production to conflict with each other, requiring the determination of a point of compromise or establishing trade-offs. For example, whilst antimicrobial use in animal feed creates increased productivity and farm income, it threatens both human and animal health through increasing the development of antimicrobial resistance.
5 Another common example is the well-established relationship proposed by McInerney
6 on livestock productivity and animal welfare: even though livestock productivity can be increased beyond maximal animal welfare (
Figure 1, point B), social acceptability is compromised by these further productivity gains due to the negative impacts on animal welfare. These points highlight the need for evidence syntheses not to be constrained to production-based and/or animal-based impacts which may only provide part of the story, but to consider environmental, economic, and human health impacts amongst others.
As a result, it is imperative that evidence synthesis be undertaken by a cross-disciplinary team with expertise across multiple fields of inquiry. In addition to animal and veterinary scientists, these teams should include agricultural economists, social scientists, information specialists, statisticians, and public health scholars, as well as experts in evidence synthesis. Without robust evidence syntheses, millions in research funding is likely to continue to be directed into projects where the answers are already known (if only we’d looked robustly!), or where findings may not actually be as useful as assumed due to subject matter myopia or lack of awareness of previous findings. In addition, lack of well-grounded research may indirectly allow negative outcomes to continue to occur with serious effects for animal welfare, human health, and agricultural productivity.
A further challenge for animal scientists is that livestock farming is heavily influenced by external factors such as weather, geography, type of production system, and labour training and availability. As an example, an intervention that improves productivity in an intensive system may not work as well or be unfeasible in an extensive system. Similarly, given different geography and climatic conditions as well as distinct breed preferences, evidence from studies conducted in Europe may be poorly translatable to farming enterprises in the tropics.
Farming is subject to significant regulatory oversight including in terms of animal welfare, veterinary practice, environmental management, biosecurity, and food safety.7 Laws are typically (but should always be) drafted against the backdrop of the best available scientific evidence,8 also taking into consideration prevailing societal and economic viewpoints. However, challenges for regulators include how to source and assimilate this scientific evidence in a non-biased fashion; how to assess relevancy and certainty in the evidence; and how to evaluate the quality and rigor of the research. It has also been acknowledged that methods in other disciplines are not likely to be directly applicable.9 A final consideration is that of transparency about the science that informs law reform agendas, which feeds into trust. If the data and methods upon which decisions are made is publicly available, others can assess the validity of the research and how that research supports the decision-making process.10
We argue that robust methods for evidence syntheses are necessary to address these issues. However, there currently is both lack of awareness and lack of knowledge within the agricultural and animal sciences about best practices for robust evidence assimilation techniques. While there has been some development into tools specific to clinical veterinary medicine, these too are limited in terms of offerings. There has been limited attention paid to methodological development around evidence syntheses, likely due to erroneous assumptions that clinical veterinary or human medicine- related tools can be applied. Below we briefly present some priorities for methodological development in this area and call for greater attention to research into these methods as well as to the fostering of cross-disciplinary networks and outlining of key considerations necessary to develop a new and robust evidence ecosystem for the animal sciences (
Figure 2).
Core Outcome Sets (COS)
Core outcome sets (COS) permit comparative effectiveness research where the benefits and harms of interventions for clinical conditions are being evaluated.11 In medicine and veterinary medicine,11,12,13,14 COS outline an agreed set of research outcomes that should be measured and reported in studies investigating a particular health condition.15 Their use allows robust evidence syntheses to be conducted, minimising wastage of research resources and supporting rapid translation of evidence to practice.16 COS are available online in a publicly available repository in order to promote their widespread development and use11.
The selection of appropriate outcomes for research study design is fundamental as any detected differences between intervention groups are considered to arise because of the intervention.
17 Outcomes operate at different levels of specificity and across domains: for example, research in production animals may consider outcomes ranging from health, production, and welfare.
18 Conceptual outcomes focus on one aspect of the domain such as meat-related parameters in relation to production. Outcomes also may be described with varying levels of granularity. For example, an operational outcome of pain may be assessed by a variety of specific outcome measures, such as cortisol levels at a fixed timepoint. In order for COS to work effectively, each component of an outcome needs to be specified precisely to enable it to be used by others (as outlined in
Figure 3).
Defining COS across a range of research areas within animal sciences will assist in building a strong body of evidence, for example about animal welfare, interventions for enhancing reproductive success or meat quantity and/or quality, or effective disease treatments. Thus, COS will ultimately lead to better translation of research to practice, less wastage of research funding through duplications and inefficiencies, and enhanced clinical decision making which will benefit animals through improved health and welfare and improved practices and profits for producers.
Standardised Reporting Guidelines
Standardised reporting guidelines ensure that published research is transparent and contains all the essential information related to the design, conduct, results, and interpretation of the study. Within veterinary medicine, there are a range of reporting guidelines available which have often been adapted from counterparts developed for evidence-based medicine. Examples include the REFLECT statement based on the CONSORT statement for RCTs,19 and STROBE-vet for observational study designs.20 The RCVS Evidence Based Veterinary Medicine (EBVM) toolkit also contains several reporting guidelines for the most common study designs.21 Inclusion of items in reporting guidelines is based on empirical evidence indicating the lack of report regarding a key design feature could either lead to biased effect estimates or would prevent evaluation of the reliability of the findings.19 Adherence to reporting guidelines ensures that key information is included, enhances reproducibility, facilitates the transferability of findings, and allows systematic reviews to perform a risk of bias assessment.
There are no reporting guidelines that are specific to systematic reviews for animal or veterinary science research. While the PRISMA 2020 statement22 can be utilised in systematic reviews in the animal sciences, these reporting guidelines have been developed for use in systematic reviews of health interventions. It has been acknowledged that the PRISMA statement is not a ‘one size fits all’ set of guidelines, with extensions being approved for systematic reviews of diagnostic test accuracy studies23 and scoping reviews24 among others. There may be scope for the development of a PRISMA extension that focuses on the unique nuances encountered in systematic reviews of animal science research.
Assessment of the Certainty of the Evidence: Moving from Evidence to Decision
The assessment of the certainty of the evidence is now considered a fundamental component of the evidence synthesis process. Determining the ‘certainty’ of the evidence involves a judgement about how certain we are that the results of research are correct to inform a decision. Established by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group, the GRADE approach assigns four levels of ‘certainty’ to a body of evidence. Levels of ‘certainty’ are specific to an outcome produced as part of a research synthesis, and in descending order, these certainty levels range from high, moderate, low, or very low. The GRADE methods have been adapted for a range of review types within healthcare. The GRADE approach provides specific criteria used to lower the certainty of the evidence (risk of bias, imprecision, inconsistency, indirectness, and publication bias)29,30 as well as criteria to increase the certainty of the evidence (magnitude of effect, dose-response gradient, and plausible residual confounding).29,31A secondary (but no less important) component of the GRADE approach addresses the complexities and challenges of moving from a synthesised piece of evidence to a formal decision. The GRADE Evidence to Decision Framework (EtDF) assists decision-makers in balancing the benefits and harms of a particular intervention or strategy against other considerations that are important to the decision-making process, but which are overlooked in primary and secondary research mediums. These considerations include contextual factors such as feasibility, equity, acceptability, and costs and resource implications. 32
GRADE methodology has also been applied outside of health care disciplines, for example in environmental health,33 public health, 34 and even preclinical animal studies.35 However, the GRADE approach is constantly evolving, reflective of the ever-changing nature of research synthesis as a discipline. GRADE methods for ‘traditional’ evidence (i.e., intervention-based research) receive regular updates and attention in the form of new guidance. Although there is some active development of GRADE for use in diverse synthesis types, these efforts are not as prominent when compared to development of the framework for more ‘traditional’ evidence sources. However, the GRADE working group recently established the GRADE for Veterinary Medicine project group, where some of these issues can begin to be addressed.
There is also scope for greater development of GRADE methods for use in diverse forms of EtDFs. Animal and agricultural sciences, and the decision-making in these fields, are far removed from decision-making processes within healthcare, and additional considerations are likely required, including societal and economic issues that may not be considered in or captured by existing GRADE EtDF methodology. Stakeholder participation is an integral part of the guideline development process to ensure that synthesized evidence is contextualized, and recommendations are of relevance to end-users. Guidance for stakeholder engagement should be adapted to an animal health setting and should include a range of stakeholders, 36 including those whose role it is to safeguard animal welfare.
Take Home Messages and Call to Action
Robust evidence syntheses in animal sciences are critical to inform future research, prevent wastage of research funds, aid adoption of research in practice, and guide policy reform.
The systematic and reproducible nature of evidence syntheses benefits law reform agendas since it provides transparency around the scientific evidence-base for a decision and assures stakeholders of this evidence base.
Methods and tools used in evidence-based medicine could be used to guide tool development within the animal sciences, but consideration is needed about discipline-specific factors such as a reliance on observational study designs, aspects of external validity such as geographic and climatic factors, and impacts that may extend beyond the animal (e.g., to humans and the environment).
In developing and strengthening techniques for evidence synthesis within the animal sciences, a multidisciplinary approach is preferable given that animal benefits may be accompanied by consequences for humans based on a one-health approach, and may be economically or societally unpalatable.
Identified deficits in the evidence-synthesis toolbox in this discipline include the lack of core outcome sets (COS), true risk of bias tools across a range of study designs, and frameworks for assessing certainty in the evidence synthesised to guide the formulation of robust recommendations.
There is a need for greater research to develop these evidence synthesis tools by academia, industry, and government. For academics, there is a need for international and multi-disciplinary cooperation to generate these tools and foster industry buy-in.
Clear strategies for dissemination and education in the use of developed tools is needed to maximise their application and realise the benefits of robust evidence syntheses for research and practice.
We propose development of a global consortium to create, host, and maintain a database on activities relevant to evidence syntheses in animal sciences, which also could drive methodological development and education efforts.
a |
There is much current work on methods to increase food availability via non-animal production using alternate protein sources. There is also controversy from some quarters around over-reliance on animal production as a food source given environment and sustainability issues. We acknowledge this but do not consider this within the current discussion. |
References
- Committee on Considerations for the Future of Animal Science Research; Science and Technology for Sustainability Program; Policy and Global Affairs; Board on Agriculture and Natural Resources; Division on Earth and Life Sciences; National Research Council. Critical Role of Animal Science Research in Food Security and Sustainability. Washington (DC): National Academies Press (US); 2015 Mar 31. 1, Introduction: Overview of the Challenges Facing the Animal Agriculture Enterprise. Available online: https://www.ncbi.nlm.nih.gov/books/NBK285726/.
- Dusel, S., Wieck, C. Evidence gaps hinder animal welfare progress in the European Union. Nat Food. 2023, 4, 348–349. [CrossRef]
- Robinson KA, Brunnhuber K, Ciliska D, Juhl CB, Christensen, R., Lund, H. Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important? Journal of Clinical Epidemiology. 2021, 129, 151–157. [CrossRef]
- Nicol CJ. A Grand Challenge for Animal Science: Multiple Goals – Convergent and Divergent. Frontiers in Animal Science. 2021,2. [CrossRef]
- Barton, MD. Barton MD. Antibiotic use in animal feed and its impact on human health. Nutrition Research Reviews. 2000, 13, 279–299. [Google Scholar] [CrossRef] [PubMed]
- McInerney, J. Animal welfare, economics and policy. 2004; 165. Report for DEFRA. Available online: https://webarchive.nationalarchives.gov.uk/ukgwa/20110318142209/http://www.defra.gov.uk/evidence/economics/foodfarm/reports/documents/animalwelfare.pdf.
- Morton R, Hebart ML, Ankeny RA, Whittaker AL. Assessing the Uniformity in Australian Animal Protection Law: A Statutory Comparison. Animals. 2021, 11, 35. [Google Scholar] [CrossRef]
- Johnston CH, Richardson VL, Whittaker AL. How Well Does Australian Animal Welfare Policy Reflect Scientific Evidence: A Case Study Approach Based on Lamb Marking. Animals. 2023, 13, 1358. [Google Scholar] [CrossRef]
- EFSA Guidance for Those Carrying out Systematic Reviews European Food Safety Authority. Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA journal. 2010, 8, 1637. [CrossRef]
- Allison DB, Fineberg HV. EPA’s proposed transparency rule: Factors to consider, many; planets to live on, one. Proceedings of the National Academy of Sciences. 2020, 117, 5084–5087. [CrossRef]
- Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke, M, Tunis, S., et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations. PLOS Medicine. 2017, 14, e1002447. [CrossRef]
- Olivry T, Bensignor E, Favrot C, Griffin CE, Hill PB, Mueller RS, et al. Development of a core outcome set for therapeutic clinical trials enrolling dogs with atopic dermatitis (COSCAD’18). BMC Veterinary Research. 2018, 14, 238. [CrossRef]
- Doit H, Dean RS, Duz M, Finch NC, Brennan ML. What outcomes should be measured in feline chronic kidney disease treatment trials? Establishing a core outcome set for research. Preventive Veterinary Medicine. 2021, 192, 105348. [CrossRef]
- Gula R, Theuerkauf J. The need for standardization in wildlife science: home range estimators as an
example. European Journal of Wildlife Research. 2013, 59, 713–718. [CrossRef]
- Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012, 13, 1–8. [CrossRef]
- Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. The Lancet. 2014, 383, 267–276. [CrossRef]
- Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017, 18 (Suppl 3), 280. [CrossRef]
- Sargeant JM OCA, O’Sullivan TL, Ramirez, A. Maximizing value and minimizing waste in clinical trials in swine: Selecting outcomes to build an evidence base. Journal of Swine Health and Production. 2023, 31, 29–35. [CrossRef]
- O'Connor AM, Sargeant JM, Gardner IA, Dickson JS, Torrence ME. The REFLECT Statement: Methods and Processes of Creating Reporting Guidelines for Randomized Controlled Trials for Livestock and Food Safety. Journal of Veterinary Internal Medicine. 2010, 24, 57–64. [CrossRef]
- Sargeant JM, O'Connor AM, Dohoo IR, Erb HN, Cevallos M, Egger M, Ersbøll AK, Martin SW, Nielsen LR, Pearl DL, Pfeiffer DU, Sanchez J, Torrence ME, Vigre, H., Waldner, C., Ward MP. Methods and processes of developing the strengthening the reporting of observational studies in epidemiology - veterinary (STROBE-Vet) statement. Accessed 31st August 2023. Available online: https://strobevet-statement.org/.
- EBVM Toolkit; Tools, Guidelines and Checklists. Accessed 31st August 2023. Available online: https://knowledge.rcvs.org.uk/evidence-based-veterinary-medicine/ebvm-resources/tools-guidelines-and-checklists/.
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews. 2021, 10, 1–11. [CrossRef]
- McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, and the P-DTAG. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA. 2018, 319, 388–396. [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun, H., Levac, D., et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of Internal Medicine. 2018, 169, 467–473. [CrossRef]
- Slack MK, Draugalis JR, Jr. Establishing the internal and external validity of experimental studies. American Journal of Health-System Pharmacy. 2001, 58, 2173–2181. [CrossRef]
- Sargeant JM, Brennan ML, O'Connor AM. Levels of Evidence, Quality Assessment, and Risk of Bias: Evaluating the Internal Validity of Primary Research. Frontiers in Veterinary Science. 2022, 9. [CrossRef]
- Centre for Evidence-based Veterinary Medicine (2021) Critical appraisal tool - Standard questions. Accessed 1st May 2023. Available online: https://www.nottingham.ac.uk/cevm/resources/our-resources.aspx.
- Barker TH, Stone JC, Sears K, Klugar M, Leonardi-Bee, J. , Tufanaru, C., et al. Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process. JBI Evidence Synthesis. 2022, 21, 478–493. [CrossRef]
- Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology. 2011, 64, 383–394. [CrossRef]
- Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz, R., Brozek, J., et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology. 2011, 64, 401–406. [CrossRef]
- Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello, P., et al. GRADE guidelines: 9. Rating up the quality of evidence. Journal of Clinical Epidemiology. 2011, 64, 1311–1316. [CrossRef]
- Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 2016, 353. [CrossRef]
- Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright, M, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implementation Science. 2018, 13, 2. [CrossRef]
- Morgan RL, Beverly B, Ghersi D, Schünemann HJ, Rooney AA, Whaley P, Zhu YG, Thayer KA; GRADE Working Group. GRADE guidelines for environmental and occupational health: A new series of articles in Environment International. Environment International. 2019, 128, 11–12. [CrossRef]
- Wei D, Tang K, Wang Q, Estill J, Yao L, Wang X, et al. The use of GRADE approach in systematic reviews of animal studies. Journal of Evidence-Based Medicine. 2016, 9, 98–104. [CrossRef]
- Petkovic, J., Magwood, O., Lytvyn, L. et al. Key issues for stakeholder engagement in the development of health and healthcare guidelines. Research Involvement and Engagement. 2023, 9. [CrossRef]
|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).