Response to New Atlantis piece about "Saving Science (from itself)"

So I came across this New Atlantis piece by Daniel Sarewitz, a long, rambling essay on how to save science. From what? From itself, apparently. And here's the solution:

To save the enterprise, scientists must come out of the lab and into the real world.

To expand upon this briefly, Sarewitz claims that many ills that befall our current scientific enterprise, and that these ills all stem from this "lie" from Vannevar Bush:

Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.

The argument is that scientists, left to our own devices and untethered to practical applications (and unaccountable to the public), will drift aimlessly and never produce anything of merit to society. Moreover, the science itself will suffer from the careerism of scientists when divorced from "reality". Finally, he advocates that scientists, in order the avoid these ills, should be brought into direct relationship with outside influences. He makes his case using a set of stories touching on virtually every aspect of science today, from the "reproducibility crisis" to careerism to poor quality clinical studies to complexity to big data to model organisms—indeed, it is hard to find an issue with science that he does not ascribe to the lack of scientists focusing on practical, technology oriented research.

Here's my overall take. Yes, science has issues. Yes, there's plenty we can do to fix it. Yes, applied science is great. No, this article most definitely does not make a strong case for Sarewitz's prescription that we all do applied science while being held accountable to non-scientists.

Indeed, at a primary level, Sarewitz's essay suffers from the exact same problem that he says much modern science suffers from. At the heart of this is the distinction between science and "trans-science", the latter of which basically means "complex systems". Here's an example from the essay:

For Weinberg, who wanted to advance the case for civilian nuclear power, calculating the probability of a catastrophic nuclear reactor accident was a prime example of a trans-scientific problem. “Because the probability is so small, there is no practical possibility of determining this failure rate directly — i.e., by building, let us say, 1,000 reactors, operating them for 10,000 years and tabulating their operating histories.” Instead of science, we are left with a mélange of science, engineering, values, assumptions, and ideology. Thus, as Weinberg explains, trans-scientific debate “inevitably weaves back and forth across the boundary between what is and what is not known and knowable.” More than forty years — and three major reactor accidents — later, scientists and advocates, fully armed with data and research results, continue to debate the risks and promise of nuclear power.

As a concept, I rather like this concept of trans-science, and there are many parts of science, especially biomedical science, in which belief and narrative play bigger roles than we would like. This is true, I think, in any study of complex systems—including, for example, the study of science itself! Sarewitz's essay is riddled with narratives and implicit beliefs overriding fact, connecting dots of his choosing to suit his particular thesis and ignoring evidence to the contrary.

Sarewitz supports his argument with the following:

1. The model of support from Department of Defense (DOD), which is strongly bound to outcomes, provides more tangible benefits.
2. Cancer biology has largely failed to deliver cures for cancer.
3. Patient advocates can play a role in pushing science forward by holding scientists accountable.
4. A PhD student made a low-cost diagnostic based inspired by his experiences in the Peace Corps.

A full line-by-line rundown of the issues here would simply take more time than it's worth (indeed, I've already spent much more time on this than it's worth!), but in general, the major flaw in this piece is in attempting to draw clean narrative lines when the reality is a much more murky web of blind hope, false starts, and hard won incremental truths. In particular, we as humans tend to ascribe progress to a few heroes in a three act play, when the truth is that the groundwork of success is a rich network of connections with no end in sight. In fact, true successes as so rare and the network underlying them so complex that it's relatively easy to spin the reasons for their success in any way you want.

Let me give a few examples from the essay here. Given that I am most familiar with biomedical research (and that biomedical research seems to be Sarewitz's most prominent target), I'll stick with that.

First, Sarewitz spills much ink in extolling the virtues of the DOD results-based model. And sure, look, DOD clearly has an amazing track record of funding science projects that transform society—that much is not in dispute. (That is their explicit goal, and so it is perhaps unsurprising that they have many such prominent successes.) In the biomedical sciences, however, there is little evidence that the DOD style of research produces benefits. In the entire essay, there is exactly one example given, that of Herceptin:

DOD’s can-do approach, its enthusiasm about partnering with patient-advocates, and its dedication to solving the problem of breast cancer — rather than simply advancing our scientific understanding of the disease — won Visco over. And it didn’t take long for benefits to appear. During its first round of grantmaking in 1993–94, the program funded research on a new, biologically based targeted breast cancer therapy — a project that had already been turned down multiple times by NIH’s peer-review system because the conventional wisdom was that targeted therapies wouldn’t work. The DOD-funded studies led directly to the development of the drug Herceptin, one of the most important advances in breast cancer treatment in recent decades.

This is blatantly deceptive. I get that people love the "maverick", and the clear insinuation here is that DOD played that role, who together with patient advocates upended all the status-quo eggheads at NIH to Get Real Results. Nice story, but false. A quick look at Genetech's Herceptin timeline shows that much of the key results were in place well before 1993—in fact, they started a clinical trial in 1992! Plus, look at the timeline more closely, and you will see many seminal, basic science discoveries that laid the groundwork for Herceptin's eventual discovery. Were any of these discoveries made with a mandate from above to "Cure breast cancer by 1997 or bust"?

Overall, though, it is true that cancer treatment has not made remotely the progress we had hoped for. Why? Perhaps somewhat because of lack of imagination, but I think it's also just a really hard problem. And I get that patient advocates are frustrated by the lack of progress. Sorry, but wishing for a cure isn't going to make it happen. In the end, progress in technical areas is going to require people with technical expertise. Sarewitz devotes much of his article to the efforts of Fran Visco, a lawyer who got breast cancer and became a patient advocate, demanding a seat at the table for granting decisions. Again, it makes a nice story for a lawyer with breast cancer to turn breast cancer research on its head. I ask: would she take legal advice from a cancer biologist? Probably not. Here's a passage about Visco:

It seemed to her that creativity was being stifled as researchers displayed “a lemming effect,” chasing abundant research dollars as they rushed from one hot but ultimately fruitless topic to another. “We got tired of seeing so many people build their careers around one gene or one protein,” she says. Visco has a scientist’s understanding of the extraordinary complexity of breast cancer and the difficulties of making progress toward a cure. But when it got to the point where NBCC had helped bring $2 billion to the DOD program, she started asking: “And what? And what is there to show? You want to do this science and what?”
“At some point,” Visco says, “you really have to save a life.”

There is some truth to the fact that scientists chase career, fame and fortune. So they are human, so what? Trust me, if I knew exactly how to cure cancer for real, I would definitely be doing it right now. It's not for a lack of desire. Sometimes that's just science—real, hard science. Money won't necessarily change that reality just because of the number of zeros behind the dollar sign.

Note: I have talked with patient advocates before, and many of them are incredibly smart and knowledgeable and can be invaluable in the search for cures. But I think it's a big and unfounded leap to say that they would know how best to steer the research enterprise.

Along those lines, I think it's unfair to judge the biomedical enterprise solely by cancer research. Cancer is in many ways an easy target: huge funding, limited (though non-negligible) practical impact, fair amount of low quality research (sorry, but it's true). But there are many examples of success in biomedical science as well, including cancer. Consider HIV, which has been transformed from a death sentence into a more manageable disease. Or Gleevec. Or whatever. Many of which had no DOD involvement. And most of which relied on decades of blue skies research in molecular biology. Sure, out-of-the-box ideas have trouble gaining traction—the reasons for that should be obvious to anyone. That said, even our current system tolerates these: now fashionable ideas like immunotherapy for cancer did manage to subsist for decades even when nobody was interested.

Oh, and to the point about the PhD student's low-cost diagnostic: I of course wish him luck, but if I had a dollar for every press release on a low-cost diagnostic developed in the lab, I'd have, well, a lot of dollars. :) And seriously, there's lots of research going on in this and related areas, and certainly not all of it is from DOD-style entities. Again, I would hardly take this anecdote as a rationale for structurally changing the entire biomedical enterprise.

Anyway, to sum up, my point is that a more fair reading of the situation makes it clear Sarewitz's arguments are essentially just opinion, with little if any concrete evidence to back up his assertions that curiosity-driven research is going to destroy science from within.

Epilogue:

OK, so having spent a couple hours writing this, I'm definitely wondering why I bothered spending the time. I think most scientists would already find most of Sarewitz's piece wrong for many of the same reasons I did, and I doubt I'll convince him or his editors of anything, given their responses to my tweets:

I'm not familiar with The New Atlantis, and I don't know if they are some sort of scientific Fox News equivalent or what. I definitely get the feeling that this is some sort of icky political agenda thing. Still, if anyone reads this, my hope is that it may play some role in helping those outside science realize that science is just as hard and messy as their lives and work are, but that we're working on it and trying the best we can. And most of us do so with integrity, humility, and with a real desire to advance humanity.

Update, 8/21/2016: Okay, now I'm feeling really stupid. This New Atlantis is some sort of scientific Fox News: they're supported/published by the Ethics and Public Policy Center, which is clearly some conservative "think" tank. Sigh. Bait taken.