Black Swan

In today’s R&D operations costs escalate dramatically as we progress a line of inquiry towards commercialization+. In the pharmaceutical industry, once we reach the clinic our costs can be has high as hundreds of millions of dollars. We are very tempted to look for ways to falsify the drug development hypothesis earlier in the process rather than later – the nomenclature in the industry is Early Project Kills+.

This is a very Popperian view of the world, in that the only truth we can know is in a falsifying truth. I can never make the statement:

All swans are white.

Indeed this was believed to be the case before the discovery of black swans in Australia. But I speak the truth when I make the statement all swans are not white as soon as I see the first black swan. This is the gist of the black swan or falsification argument. The only valid truths are truths about negatives.

However, falsification is just a parlor trick: I can change my generalization+ of what it takes to be a swan to exclude that black creature. It’s not really a swan. This process of changing my generalization is called ad hoc hypothesis generation+. I subtly, and continuously, adjust my hypotheses to account for falsifying instances. And given the creativity of the human imagination, there’s no limit to the number of ways I can repeat this ad hoc cycle.

Consider, for example, the statement:

In all cases of Fibromyalgia a patient feels pain at 11 or more points when force is exerted at 18 points.

What about a patient with pain in only 10 out of the 18 points? What does feels pain really mean? How about 11 out of 19? We can change the criteria to allow this patient in; we can let the patient know that our threshold for pain is lower for certain points; we can diagnose the patient as being in an early stage of the disease. Trial lawyers are not so persnickety about pain when building a class action suit.

Falsification encourages the rash belief that it’s easier to know when to stop a line of inquiry – to the huge detriment of interpersonal relationships, personal drive and ambition. Industry participants argue that the weight of falsifying observations will eventually collapse an hypothesis. I disagree. I’m a believer in the infinite creativity of our ad hoc hypothesis creators. It is just as hard to prove an hypothesis false as to prove it true, at least until we try the drug in humans. This is one of the basis for the theme of this section: you don’t know anything until you know it in humans.

In general, falsification is not helpful in drug development because we rarely use the qualifier all, as in this drug helps all patients or this drug is safe for all patients. Instead we deal in probabilities. There are always falsifying instances for the effectiveness of every drug: ‘…drug has been found to be not useful for patients with X or Y disorders.’

Falsification is always useless with drug safety issues. If we do our job right and assuming it’s our drug that causes the safety issue, there will be very few instances confirming our drug has a safety issue. If anything, falsification as an approach in safety issues points towards the wrong answer.

Black Swan or falsification thinking does give one useful insight. We tend to recruit homogenous patients for early R&D. It’s like only searching in England for swans.

Pfizer’s drug Tremelimumab, anti-CTLA-4 monoclonal antibody for the treatment of patients with advanced melanoma had exceptional results (interocular+ results) in early Phase 1 and 2 clinical trials, but a review of interim data from the pivotal Phase 3 trial found it would not demonstrate superiority to standard chemotherapy, leading to the decision to discontinue the study. We tend to only take one scouting mission, called pivotal Phase II trials in the pharmaceutical industry. It’s time to start looking in Australia.

See falsifiability for a picture of an Australian black swan.