You Get the Answer You Want

It all depends on how you ask the question.

A seminal article in the behavioral field Resolving Scientific Disputes… describes a scientific bake-off used to resolve differences in findings between researchers Gary Latham and Miriam Erez, with Edwin Locke as the referee. The topic was an old chestnut, whether having employees participate in annual goal setting builds their commitment to and attainment of (mechanical) goals. Latham and Erez studied the topic and came up with opposite findings. The bake-off was a controlled series of studies to reconcile the differences. To anyone who pays attention to the manipulations of public pollsters, the findings from the bake-off are not surprising: it depends on how you ask the question.

The topic of the bake-off is not important for our discussions, but the findings on the value of the bake-off itself are quite relevant:

  • Very subtle differences in study procedure, or the sum of a number of small differences can lead to dramatic differences in outcomes
  • A trusted mediator gives antagonists in the bake-off the mental license to review and question the influence of their own activities on the results. The context of the study greatly influences results and very often researchers are unaware of these influences because they are part of the context. They have no external reference point.
  • A mediator will not be able to help when the antagonists are at odds because of philosophical differences – each antagonist must be incented to seek the best answer even if it undermines their pet theory or core beliefs.

To bring this home for R&D in the life-sciences industry, you get dramatically different outcomes using the same protocol and clinical trial design depending on how you procedurally conduct patient recruitment and study participation. The context of the patient interaction can be as important as the actual physical interventions. Soft music and the smell of herbal tea change the context (as an extreme example). Obviously context is more relevant to the treatment of mental illness than bone fractures, but there are often measurable physiological changes (e.g., dopamine levels in the brain) due simply to the context: the statement stands.

If you know these subtle differences can make or break a clinical trial how would you change the way you run the trials? Suppose we’ve just spent hundreds of millions of U.S. dollars to develop a new drug and you’re getting ready for Phase III clinical trials. Do I just throw my drug to the lowest-cost supplier of clinical trial services and hope for the best? That’s how it’s done today. What if I had in my back pocket an understanding of the subtle differences in study procedure that could swing the results my way? How would I get them?1 How would I use them to influence the results of the upcoming trials?

Do I write down the subtle differences in a formal document and hand them to my study managers? No. The purpose of clinical trials is to replicate conditions to be seen in patient populations once the drug is marketed. We take laboratory results and conflate them to a prediction of what’s to be seen in the field (see here). If our study protocol calls for an investigator who is amiable, buxom or attractive to the patient, then we must show how this amiability will be replicated once the product is on the market.

A strictly mechanical or robotic approach to clinical trials will not work. Instead we pursue clinical trial management at a very personalized level. I insert my understanding of the subtleties by enriching the investigators and patient recruiters to my advantage There are a thousand subtle ways to nudge the context of the clinical trials in a favorable direction. Be careful. Individuals (i.e., employees or contractors) have great difficulty in discerning the subtle differences between the intent of this advocacy and fraud.

Overt manipulation of clinical trial results is unethical and can be illegal. But simply positioning a drug to show it in its best light is an obligation we have to all who came before us. Recall Phase III pivotal trials are mostly Cover-Your-Backside exercises that add no further scientific understanding to the drug. We grant you ethical license to be more creative. Don’t let others draw arbitrary lines; look for guidelines or cautionary tales instead. Many individuals (i.e., the quants, the in-house lawyers, federal regulators) conflate the mechanics of running a clinical trial with the ethics of trial management. Instead we look for a balance.

A firm can only achieve a balance between ethics and results by building commitment to and attainment of (creative) goals by its employees. We set the proper context for the work in R&D: on how we ask the question of employees or contractors. Done right, the employee will be committed to doing what it takes (legally) to get this drug to market. Done wrong and you open yourself up to whistle-blower lawsuits from disgruntled ex-employees. There are no simple answers.


Editor's Picks for October, 2010

  • 1. The answer is rarely simple. We seek subtleties that only affect patients taking the drug and not the placebo. We look to Ac-Cent-Tchu-Ate the Positive of the pharmacological effects of the drug. If individuals act differently simply because they know they're in a trial (the Hawthorne effect) how do we play this insight to our own advantage?