Tiers of Evidence

...objective, subjective, and political. Researchers must demonstrate an understanding, an acceptance of the fact of life, that objective (empirical) evidence only carries you so far.

Having a robust culture of Critical Thinking+ in R&D is just the start. Objective evidence, the domain of Critical Thinking, refers to empirical evidence: evidence supported by data and unfiltered by researcher preconceptions. What does the data tell us? Intuition and judgment picks up where objective evidence leaves off. There is never enough time to gather all the facts and a good implementation of Critical Thinking acknowledges this limitation. Critical Thinking deals with objective evidence, but recognizes the contribution of intuition and judgment into the analysis of evidence.

Political sensibilities sits as the capstone of evidence. The only evidence that matters is the evidence as it is perceived by the funding agents – regulators. Our job is to take raw empirical data and to weave it into a credible narrative for the uninitiated. Whether or not gathered data is acted upon in no way passes judgment on the inherent quality or usefulness of the data - only its political value. Politicians rightly ignore or selectively filter evidence to support their claims. The end game+ for evidence is not to come up with a universal claim to truth, rather to come up with a convincing case for continued funding, which in turn depends on the trust funding agents place in the quality and usefulness of the evidence they receive. Since it is often the case that funding agents cannot independently verify the quality of the evidence, they rely on trust in those who deliver the evidence. Delivery agents do everything they can to build and maintain the trust of the funding agent, and to shape their interpretation of the evidence.

This is the concept of tiers of evidence+. Objective evidence only carries us so far. It is only when a wider audience (e.g., funders, regulators, interest groups) has developed a conception of our objective evidence, an interpreted view that we helped them construct, that the final political 'truth' emerges. The objective evidence is a major player in this conception, but there is often a great deal of uncertainty. We can’t test for every eventuality. Limited, fragmented and often contradictory facts must be fitted into a likely narrative. This is the role for judgment, intuition and political sensibilities.

Further Reading