Why does push-button science push my buttons?

We just did some single cell RNA-seq using the Fluidigm C1 machine. Seems fancy, right? Sequencing from a single cell? Actually, it was remarkably simple. We just put some cells in a tube and squirted them into a chip, put the chip in a machine, and said go. Okay, maybe a little more than that, but not much. For a while, something about this really irked me about this, and I’ve been trying to get to the root of that feeling. Here’s what I realized about myself.

We, like many others in imaging, do artisanal experiments. Our experiments take patience and some degree of debugging and know-how, and getting a good data set still requires some amount of care and attention. Although we have automated much of our workflow, we most definitely do not have a platform. And the fact is that I like it that way. I take pride in high quality data, in experiments that not everyone can do, crafted by hand by highly skilled experimentalists. I love knowing the ins and outs of all the little details that go into interpreting an image, in figuring out what may have gone wrong and how best to fix it. And maybe I’m all wrong for feeling this way.

In the end, isn’t it a good thing if experimental methods are commoditized, made robust and broadly available? If it’s so easy to do something that anyone can do it? That we are freed to spend more time thinking about science and less time mindlessly pipetting clear liquids from one tube to the next?

I think a part of what bothers me is fear. It’s like I’m a scientific luddite, scared that there will not be a place for our type of work in the future. It’s a fear that’s probably largely misplaced (I hope). Of course, one issue with these push-button machines is that the big, fancy labs typically have access to them well before anyone else, and so they get their big, fancy papers out first. But those papers generally tend to be just about the least imaginative and most obvious thing you can do with this tech, as though at the end of the run, somebody pushed the next button over labeled “Write paper”. Our job is to think of something clever to do with these new capabilities and use it to enhance our existing strengths. In that way, the more things change, the more things remain the same.

Some pics of the lab from our good buddy Marc Beal

Marc Beal, Stellaris RNA FISH champion numero uno, came to visit the lab and took some great pictures at our traditional BBQ 'n Beal outing at Baby Blues BBQ:

One must enter the secret room through the secret doorway:

Our guides on this magical journal:

Let the games begin:

Some cows were definitely harmed in the making of these pictures:

Thanks for a great time, Marc!

How to get people to do boring stuff they don’t want to do in lab

Lately, we’ve been working on a lot of infrastructure and process-oriented aspects of our work in lab, like a complete overhaul of our RNA FISH analysis software (now in sufficiently good shape to be publicly available to everyone), a probe database, and thinking about how best to organize our growing RNA-seq datasets. Once we have established what we believe to be best practice, though, the next issue is compliance. It’s one thing to tell people what they should do, quite another to actually get them to do it. For instance, we can all say “if you’re going to analyze your RNA-seq data, you should use this data organization scheme”, but there’s a natural entropy at play when people actually do work in the lab, and non-compliance is a natural by-product. How can you enforce best practice?

Well, actually, before starting to think about enforcement, I think it’s worth making sure that whatever scheme you put in place has actual, real benefits to people in the lab. I’ve come to realize that process, while it can enable science, is not science in and of itself, and it’s not always worth the effort. It’s a fine line, and perhaps somewhat a matter of personal taste; I think some folks are just fussier about stuff than others.

So what are the benefits? For our lab, I feel like there are three main benefits to building process infrastructure:

1. Error reduction: To me, the most useful benefit to having a standardized and robust data pipeline is that it can greatly reduce errors. The consequences of mixing up your datasets or applying the wrong algorithm can be absolutely devastating in a number of ways.
2. Reproducibility/documentation: For data, I feel, as do many others, that it is imperative to be able to reliably (and understandably) reproduce the graphs and figures in your paper from your raw data. Frankly, in this day and age, there’s no excuse not to be able to do this. Documentation is just as important for other things we do in lab, whether it’s how we designed a particular probe or what the part number is for some kit we ordered 3 years ago and is about to run out.
3. Saving people time and facilitating their work: Good infrastructure can save time in a number of ways. Firstly, it hopefully leads to less wheel-reinvention, which I’ve seen all the time in other labs. Another way it saves time is by (hopefully) leaving a data trail; i.e., “That data point looks funny, can you show me the image it came from?” Good infrastructure makes it easy to answer that question, and makes it much easier to explore your data in general. If getting answers is easier, you will ask more questions, which is always a good thing.

So what’s the problem? Well, for points 1 and 2, the issue is that error reduction, reproducibility and documentation are just not that exciting, at least not to people who are more interested in doing science. That, and the payoff is typically a sigh of relief a couple years down the line. My experience thus far has been that most systems for documenting lab stuff, no matter how sound the rationale, just don’t stick without some serious effort. For instance, we have a probe “database” (i.e., spreadsheet) that is woefully out of date. And we have a number of protocols that are fairly out of date, and an orders spreadsheet that is out of date, you get the idea. Same for RNA-seq and RNA FISH datasets, at least at high level data organization. You know the feeling: “No, not that transcription inhibition dataset, that’s the one that came out funny because of the cells acting weird, use this one instead…” The only way to enforce in these cases is to create a punitive rule, something like "no more orders placed until you update the ordering sheet". Sucks, but I guess that works.

But point 3, saving time and facilitating work, that’s something everyone can all get behind without any prodding. And then there's never any issue of compliance. For instance, our software provides all the backend to make sure that our data is fully traceable from funny outlier data point to the raw images of a particular cell. But it also provides all the tools to analyze data and use all the latest tricks and tools for image analysis that we have developed in the lab. For this reason, it's essentially inconceivable that anyone would spend any time writing their own software and doing anything else, the benefits are big and, importantly, immediately realizable future.

So what I’m thinking is that we somehow have to structure all the boring lab documentation tasks so that there is some immediate gratification for doing so. What can that be? I’m not sure. But here’s an example from the lab. We’re working on having our probe database automatically generate identifiers and little labels that we can print out and stick on the tube. Not a huge deal, but it’s sort of fun and certainly convenient. And it’s something you can enjoy right away and only get if you access the probe database. So I’m hoping that will drive the use of the database. A more ambitious plan is to develop similar databases for experiments and consequent datasets that would enable automatic data loading. This would be both important for reproducibility, but would also be enormously convenient, so I’m hoping people in the lab would be excited to give it a whirl.

Anybody else have any thoughts about how to encourage people to participate in lab best practices?

Brilliant post by Casey Bergman about why science is getting harder

I just was browsing through Casey Bergman's blog, and I found this fantastic post about growth in the scientific "industry". The basic point is that the growth in the number of scientists is an exponential trend that is increasing faster than the rate of overall population growth, and perhaps the general crunching sound that we're experiencing in science right now is a result of a transition from exponential growth to saturation. Feels like a very sound argument to me!

What are technology transfer departments for?

I just wrote a post about how Penn's tech transfer policy is worse than usual in terms of rewarding inventors, and Sri Kosuri (all around cool guy and now running his own show at UCLA) made the excellent point that most tech transfer offices don't actually make money, except for a few that struck gold with a blockbuster drug or something. Very true! Which got me thinking: what is the mission statement of the tech transfer office? And the answer I came to is that I have no idea.

Could it be making money for the university? It's true that overall, tech transfer is not a money making venture. So it would seem that the university is on its face not interested in making money on tech transfer, and that it's provided as a service for the faculty. Partly true. However, there are definitely rumors at places that they overhauled tech transfer after screwing up some big blockbusters. So the university is definitely interested in perhaps winning the lottery. In fact, this is not so very different from the overall business model of undergraduate education at many of these same places: they somehow lose money on each student (despite the astronomical tuition), making it up when the occasional graduate becomes super rich and donates a ton of money back to the university. So my feeling is that the university definitely believes that there is some potential financial gain from tech transfer, and would like to make sure their tech transfer office doesn't miss something big.

That said, it does not really make money, and I certainly don't think tech transfer is typically run with a profit motive in mind most of the time. The idea, in theory, is to get the ideas from our labs out into the public domain. Wait, don't papers already do that for free? Yes. But companies typically would actually prefer to license the technology, since it can give them a competitive advantage in the marketplace. So if you want to get your tech out there, you really need some IP, no matter how altruistic you are. In practice, though, I'm not sure what criteria they use for deciding what to patent, and its an important choice because if they choose to patent, then it's a serious cost borne by everyone, which is what I was complaining about before. I'm guessing that they pursue more stuff than a more profit-minded entity would, and that's fine–at least some faculty can get some line items for their CV.

I see some problems with this, though. I have only very limited experience with this stuff, but I just don't see much coming from filing patents and having them sit on some shelf in the tech transfer office waiting for a wandering Google search to bring it to life. To the contrary, from what I've seen, getting your technology out into the commercial world requires a lot of care and attention and fostering of ties to industry partners (to its credit, Penn is making a concerted effort in this regard by creating an "innovation center" for startups and the like). It's a serious amount of work to establish and maintain these ties, and it doesn't just happen on its own. Its something that you have to want to do, and not everyone does.

Which brings me to another point: is the point of tech transfer to enable us as faculty to profit from our inventions? Again, not sure. They sure do tax the hell out of it: as I mentioned earlier, you're only getting around 22% of the royalty income, at Penn at least. Now, I griped about how little we get earlier–what roils me most is Penn compared to other places–but there's really not much I can do about it, and its not going to prevent me from trying to commercialize our work. But there is a line somewhere, and we're not all that far away from it. For instance, if the percentage we got was 1%, I think it would be pretty hard for me to justify all the time I would spend on commercialization if there was so little to show for it at the end of the day. What about 5%? 10%? 22%? Dunno, all I know is that there's a line in there somewhere. Of course, if you don't do any work, then I suppose any payout is fine, but on the other hand, like I said, those patents are unlikely to go anywhere. (By the way, remember that this is a percentage of the royalty income, which is already a very small percentage of sales, and then that gets split with various inventors, etc.)

All this to say that I'm still not sure exactly what the university hopes to get out of tech transfer, either for itself or the faculty involved. Solvency may be a goal, or may not be; empowering faculty may be a goal or may not be. In the end, though, I have to say that as I'm writing this, I'm more and more excited about the idea Penn has of creating tech incubators with resources to help take technology to the marketplace. Perhaps that's a far better model for empowering faculty to commercialize their work than people like me quibbling about a few pennies here or there.

ANOTHER NIH Biosketch format?!?

NIH is now set to introduce yet another Biosketch format. And this one requires yet more work, requiring investigators to write (subjectively) about their primary contributions to various fields for up to 5 pages (5 pages!). Ugh, can we just settle on a simple CV and leave it at that?

Penn’s patent policy is crummy for inventors

So Penn has decided to invest heavily in “innovation”, whatever that means. I think part of what it means is transferring technologies developed in academic labs at Penn to the outside world, which is of course a good thing all around. Of course, the devil’s in the details. And Penn’s details are devilish indeed!

On their fancy new website, Penn says “[Penn’s patent policy] provides a means for Inventors to receive a generous share of any income that is derived from the licensing of inventions they create as Penn employees…” Which is funny, because frankly, of all the places I’ve been, it is by far the least generous towards inventors.

For those of you who are not particularly familiar with patents and licensing at universities, here’s how it usually works. Basically, the university owns everything you invent. You own nothing. Fair or not, that’s the way it works. If you have something that you think the outside world would find useful (and is patentable), then you go to the university’s technology transfer office. Naively, the idea is that these folks will decide on whether your work is patentable and worth patenting. They then shop this "intellectual property” to various companies/startups to try and strike a licensing deal. In practice, the issue is that most of these patents just end up sitting on the shelf and never get licensed by a company. It turns out to be very hard to just have a patent sit there and somehow find a home. I think that commercialization works best if the professor herself is actively engaged in trying to develop commercial interest in her technologies, either by making a startup or getting an existing company interested–relying on someone else to make those connections in a largely anonymous fashion seems like playing some very low odds.

The issue is that all these unlicensed patents still cost money to prosecute, and somebody’s got to pay for it. And that’s where Penn’s system gets particularly bad. Typically, the breakdown is such that the inventors get around 1/3 of the money from licensing (the rest goes to various factions at Penn). Some places its a bit more (I’ve seen 35%) and some a bit less (Penn is stingy at 30%). But then come all these legal costs, and here Penn does something very strange. They basically take all the legal costs and take it primarily out of everybody’s personal share. These legal costs typically amount to a whopping 25% of the total inventor’s share. So effectively, you’re getting just 22%! And, more importantly, this applies even if you have already gotten the licensee to pay all the patent expenses! That means that even though the company you are working with is paying all the legal expenses, you are still paying out of your own pocket for everyone else’s patenting costs. In other words, the winners pay for the losers, and they pay a lot.

At other places, they will charge non-legal operating expenses out of the initial pot of licensing income, and then have you pay back the legal expenses on your own particular patent out of the royalties first. This way, if your licensee pays your legal expenses, that directly benefits you. This makes some sort of sense, I suppose–I’m still not sure why inventors get so little, but at least it more directly rewards those who actually manage to get their work out there. Note that somebody still has to pay for the unlicensed patents, but at least those costs usually get split between the university’s share and the inventor’s share, which lightens the load (especially since the university share is much bigger!).

Anyway, look, realistically, nobody’s getting into academia to get rich. That said, commercialization is the best way to have a real impact in the world–that’s just a fact. Why shouldn’t we as scientists benefit somewhat from the ideas that we work so hard to cultivate? And why should Penn scientists benefit considerably less than those at other universities?