In a recent paper published in BMJ, researchers proposed that the tools and frameworks of mechanistic evidence, sometimes known as evidence-based medicine (EBM) + combined with traditional EBM, could help overcome the very protracted coronavirus disease 2019 (COVID-19) pandemic.
Study: Adapt or die: how the pandemic made the shift from EBM to EBM+ more urgent. Image credit: CKA/Shutterstock
EBM+ includes conceptual tools and quality frameworks from various fields, including complexity science, engineering, and the social sciences, and a more pluralistic approach to obtaining “high-quality” mechanistic evidence.
background
Traditional methods, such as randomized controlled trials (RCTs) and meta-analyses, contributed significantly to the science of COVID-19. However, historically these methods have answered simple and focused questions in a stable context, such as those in biomedicine. They have significant limitations when extended to complex situations. In this case, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a new pathogen, caused chaos in multiple sectors, medical, social, economic and political dimensions in a global context of rapid change.
SARS-CoV-2 has proven to be tenacious and changeable. While RCTs and meta-analyses of RCTs remarkably overestimated the effectiveness of drugs and vaccines against COVID-19, some scholars argue that addressing some aspects of it required mechanistic evidence critically. COVID-19 led to complex situations where multiple variables interacted dynamically with great uncertainty.
Decisions had to be made in days (urgency), not years, and the consequences of not acting would have been catastrophic (threatening). In these situations, dismissing mechanistic evidence and overvaluing poorly designed or irrelevant RCT findings cost thousands of lives. Overall, the COVID-19 pandemic presented an epistemic opportunity to understand, debate, and adopt EBM+.
About the study
In the present study, the researchers argued that although the hierarchy of evidence that places mechanistic evidence at the bottom and meta-analyses of RCTs at the top is a useful heuristic, it is not applicable in all circumstances . There should be sufficient guidance to apply the hierarchy flexibly depending on the nature of the research question and the extent of complexity involved.
In fact, RCTs are not a panacea; therefore, systemic reviews are crucial for interpreting primary evidence along with modifications to the hierarchy of evidence to answer contemporary policy questions regarding SARS-CoV-2.
In addition, the researchers suggested modifications to the hierarchy of evidence for mechanistic evidence. They emphasized combining mechanism evidence with probabilistic evidence from clinical trials and nonrandomized studies, both comparative and observational, to make a strong case for causality. It is because a plausible mechanism confirms or denies the stability of the causal relationship between environments.
Such an exercise could ensure that public health interventions work, as they involve both the upstream causes and the ’causes’ through which the interventions might work. For example, the causes of COVID-19 included family structures and their interactions, economic and cultural variations, etc., and personal attitudes, beliefs, capacities, and resources determined whether public health measures worked. Furthermore, the researchers argued that mechanistic evidence is inherently explanatory.
Early in the pandemic, scientific research into COVID-19 was prematurely fixed and limited to a population intervention (or exposure) comparison of outcomes (PICO) format that reduced pre-defined outcomes and suppress the scientific imagination. These studies estimated infections in mask users in statistical terms. However, they did not account for the unique spread patterns of SARS-CoV-2 (eg, overdispersal, indoor predominance), all of which could have pointed to its predominantly airborne transmission .
A randomized RCT advised 6,024 people to wear masks outside the home. Their results showed SARS-CoV-2 infection in 42 and 53 people (1.8% and 2.1%) in the intervention and control arms, respectively, which is statistically insignificant. However, supporters of EMB argue and evidence that wearing masks in public does not substantially reduce infection.
The Cochrane Database of Systematic Reviews encouraged policy makers to adopt the precautionary principle based on the findings of this flawed study because of the urgency of the situation. However, an EBM approach would have taken into account studies elucidating the mechanisms by which masking might work. For example, tests of:
i) real case studies and mathematical modeling studies supporting the airborne transmission route of SARS-CoV-2
ii) Engineering studies showing the filtration properties of different masks
iii) Psychological and sociocultural evidence of whether people wear masks
Carrying out public health interventions and ensuring that they are implemented, for example the face mask, is a complex phenomenon. It has multiple components that act interdependently and individually at multiple levels. Masks could be homemade or produced according to formal technical standards. People can mask themselves as expected by society, the organization or the legal order.
In general, their proper implementation is complex in real-world settings for multiple social and behavioral reasons. Thus, complex systems require a new paradigm with designs that can capture dynamic changes while simultaneously adapting to nonlinearity and embracing uncertainty. In support of structural evidence, researchers discussed how engineering designs take into account physical and technical properties and capabilities, social needs and impact assessment with the example of personal protective equipment ( PPE).
The use and impact of PPE is influenced by indoor air quality. However, an RCT comparing these products to less effective protection to “prove” their value in protecting against chemical contamination in a lead smelter is meaningless. All that is required is robust certification systems, standards and workplace use protocols for PPE. The PPE in use today has already proven to be a boon against occupational hazards for millions of healthcare workers worldwide.
Professor Susan Michie, adviser to the UK’s Scientific Advisory Group on Emergencies, proposed the capability-opportunity-motivation-behaviour model. This model helped explain why people prefer or not to wear a mask. In some cases, they may lack key knowledge (eg, they may not know whether cloth masks are as effective as surgical masks) or they may not physically want to because they don’t fit well. Similarly, motivation (e.g. conscious beliefs) or opportunity (e.g. employer bans masks) may be lacking. However, failure to explore these broader influences could lead to misleading conclusions from experimental studies.
Conclusions
In the present study, the authors placed more emphasis on other sources of evidence, particularly mechanistic evidence, to manage the complex and rapidly changing COVID-19 pandemic. As the EBM+ debate continues, they hope to contribute more articles on how an EBM+ approach could improve the contribution to pandemic science.