Report

Teaser, summary, work performed and final results

Periodic Reporting for period 3 - PhilPharm (Philosophy of Pharmacology: Safety, Statistical standards and Evidence Amalgamation)

Teaser

The project aims to develop new standards of evidence for causal assessment in pharmacology with a special focus on harm assessment. For this we have developed a system for evidence amalgamation based on an epistemic Bayesian network. In particular the project has been carried...

Summary

The project aims to develop new standards of evidence for causal assessment in pharmacology with a special focus on harm assessment. For this we have developed a system for evidence amalgamation based on an epistemic Bayesian network. In particular the project has been carried out until now along three tightly interconnected strands of research:

I. The development of a theoretical framework that responds to the three main project objectives: 1) a foundational analysis on statistical/causal inference with a focus on the critical assessment of current practices in drug approval and pharmacosurveillance; 2) a unified epistemic framework within which different kinds of evidence for pharmaceutical harm can be combined and used for decision: evidence amalgamation; 3) a theoretical framework for the development of new standards of drug evaluation. These objectives have been jointly pursued by developing a Bayesian framework for causal assessment in pharmacovigilance, drawing on the philosophical literature on causation, foundations of statistics, formal epistemology and at the same time directly involving Drug Agencies across Europe;

II. A theoretical analysis of the assumptions implicit in our Bayesian framework with a special focus on higher order dimensions of evidence, such as reliability, coherence, consistency and variety of the body of evidence, and with this, the development of a Formal Epistemology of Medicine.

III. Analysis of computer aided approaches to knowledge management in the medical setting (computational modelling and simulation, machine learning tools), in view of incorporating this kind of data into the overall evidence appraisal.

Work performed

I. By addressing the debate on statistical and evidential standards in medicine we have developed a new model for causal inference that incorporates concern from various sides of the debate (reliability of causal inference, optimisation of the available evidence, clinical and external validity of results). This is a Bayesian network grounded on the theoretical framework presented in (4) and further developed in (19).
In our Bayesian net, the causal hypothesis is the root node, its children are (imperfect) indicators of causation (derived from the Bradford-Hill guidelines for causality; Bradford Hill A. (1965) The Environment and Disease: Association or Causation? Proc R Soc Med. 1965 May; 58(5): 295–300).
Concrete data, study reports of heterogeneous kinds (randomised clinical trials, epidemiological studies, animal experiments, cellular, molecular, and genomic data) feed into the different causal indicators.

From an inferential point of view, the framework exploits the coherence of heterogeneous evidence in order to obtain a probabilistic assessment of the hypothesis of causation between drug and (side-) effect. This both responds to new evidential requirements in pharmacovigilance and to the daily needs of pharmacovigilance practice. For this reason we are also collaborating with Drug Agencies (especially the Uppsala Monitoring Centre –WHO, the Austrian Drug Agency AGES, the Italian AIFA, and the German BfArM) in order to integrate experts’ insights in our tool. Visits to some of the agencies have already taken place and a second series will follow, that will enable us to further develop our framework into an implementable instrument.
By breaking down the different dimensions of evidence, strength, relevance, and reliability, E-Synthesis allows them to be explicitly tracked in the inferential process, in that it makes it possible to parcel out the contribution of each dimension. This also allows one to incorporate a higher order perspective on evidential support by effectively embedding these various epistemic dimensions into one inferential tool.
We are planning to submit a Proof of Concept for the development of a software solution based on our theoretical framework, also in cooperation with the Agencies.

II. The second area of research analyses the foundational issues related to our model for evidence amalgamation. In particular, we examine scientific inference with the lenses of Bayesian epistemology, philosophy of statistics, and philosophical theories of causation and causal inference. This line of research resulted in our main paper (4) and its sequels (2, 3, 9, 10, 13, 14); foundations of Bayesian inference and methodological applications have been mainly analysed in 5-7. A series of papers has led to various results concerning the so called “Variety of Evidence Thesis”: that is, the (debated) statement that ceteris paribus, more heterogeneous evidence coming from independent sources is expected to be more confirmatory than less varied evidence: 20-22.
Regarding this latter line of research, we show in (20) that Bovens and Hartmann’s (2003) results concerning the failure of the variety of evidence thesis (VET), mainly rely on the unreliable instrument being a randomiser, and of a very specific kind. Furthermore, Bovens and Hartmann\'s results run against the “too-good-to-be-true” intuitions underpinning suspicion of bias for considerable long series of reports from the same testing instrument. In order to account for the “too-good-to-be-true” intuition, and for the related suspicion of systematic bias, we developed a model where the instrument may either be reliable but affected by random error, or unreliable and systematically biased towards delivering positive reports (but non-deterministically so). More generally, within this line of research we are trying to develop a formal model of scientific inference that takes into account higher order dimensions of evidence such as the coherence

Final results

The Bayesian network developed for probabilistic assessment of causal hypotheses, “E-Synthesis” is a total novelty in the field. Whereas standards Bayesian network for causal assessment are intended to be used for diagnosing token causation (causal diagnosis for a given individual or event) and take concrete facts as input (“blood pressure”, or specific symptoms, which have been associated with the disease), our system for causal assessment addresses inferences of general causal laws (e.g. “Does Paracetamol increase the risk of Asthma in children?”) and takes as input data from experimental or observational studies at the epidemiological, clinical, molecular and genetic level, from heterogenous sources (spontaneous reports, clinical studies, animal studies etc.). This methodology is also radically different from standards meta-analyses, where input studies must be homogenous both from a methodological point of view, and also regarding the kind of population sampled.
Our framework may be characterised as a way to justify intuitions about causation often present in narrative reviews, where the coherence of data from basic science, epidemiologic studies etc. is taken as confirmatory - although inconclusive - evidence for causation. Furthermore, our framework allows one to provide at least a formal frame for expert reasoning, and at most a possible quantification of the strength of the evidential support available, in view of making timely decisions regarding drug circulation in the market.

We plan to further develop our formal approach by adopting a game-theoretic perspective on evidence in medicine and pharmacology.

Website & more info

More info: https://philpharmblog.wordpress.com/.