tag:blogger.com,1999:blog-55061357185333667642024-03-12T23:35:50.607-04:00RajLabRandom musings from the Raj Lab for systems biologyARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.comBlogger291125tag:blogger.com,1999:blog-5506135718533366764.post-60453865364527718762024-02-05T10:35:00.002-05:002024-02-05T10:35:40.150-05:00Pre-registration in molecular biologyA few years back, perhaps in pre-pandy times, I was on a faculty development panel in which I was one of two presenters. I was of course there to present on how to use Twitter to build your brand (sigh, I’m lame), and a more senior faculty member (I think a neuroscientist) was there to talk about pre-registration in lab work. He was very kind and wise-seeming, and explained how he had been pre-registering their results in the lab for a while, and how it transformed their work.<br /><br />What is pre-registration? It’s probably most familiar to you in the form of clinical studies, where there was a notorious selection bias in which results would be reported. Like, does drinking coffee cause flatulence? One would have to do a randomized controlled trial to check. But if people did, say, 100 clinical trials and only reported the ones where there was a “positive” result, then you would see 5 clinical trials with p < 0.05 showing that coffee causes flatulence, and none of the contradictory results. So now you have to pre-register a trial, meaning that you have to say, I am going to do this trial with this power and what not, and then you are obligated to report the outcome, no matter what the outcome is. A great idea!<br /><br />But here was someone advocating for pre-registration much closer to home, in our day to day lab work. I remembering being vehemently and vocally opposed. Sure, clinical trials are one thing, with a clearly stated hypothesis and major resources devoted to a single experiment. But in my line of work, where we are constantly trying new experiments and checking out new avenues of work, where there are tons of false leads and new directions? How could that possibly work without gumming up the works in needless bureaucracy? I was vehemently and vocally opposed, to which the senior faculty member just patiently and calmly responded “Sure, I hear you, just think about it”.<br /><br />Ever since, I keep coming back to that moment, and it has come to have a major effect on how I approach our science—and especially our reporting of it. The key take home point is: <b>if you did an experiment to answer a question, and you don’t have any reason to exclude it based on the experiment itself, then you have to report the results.</b> Repeat: <b>unless there is an independent basis for the exclusion of a result, you have to report the results.</b> Or, to put it another way: <b>if you would have included the data if the result had come out the other way, you have to report it.<br /></b><br />Selective reporting of data is a strange issue in molecular biology in that almost everyone agrees that it is wrong and yet the overall culture of the field leans towards selective reporting in so many ways. Here is an example from our own work. In a recent paper, we were trying to confirm the knockdown of a particular protein. We were able to show a convincing knockdown by RNA FISH, but also wanted to show that the protein levels went down. We did a bunch of westerns, but the results came out ambiguously: sometimes we saw an effect and sometimes not (there are reasons that that could be the case, but we didn't confirm those because they were very difficult). The standard thing to do here would be to not report the western results. But there was no reason to exclude the experiment other than being annoyed with the results. So, we reported it.<br /><br />But again, the cultural standard in molecular biology is often not to report such ambiguous results. I saw this mindset a lot early in my career, back when RNA FISH was considered cool and people wanted our help to add some RNA FISH to their paper to spice it up. There were several times when people came to us with data in support of a, shall we say… “fanciful” hypothesis, and then we would do the RNA FISH, which would basically show the hypothesis was wrong. At which point, the would-be collaborator would beg out, saying that given the “ambiguous” nature of the RNA FISH results, “perhaps we should save the data for the next paper” (which of course never materialized). After enough of these moments, I started asking potential collaborators what stage of their paper they were at, and if they were close to the end, whether they really wanted us to do this experiment. At least one time, when faced with this choice, the person said, uhhh, let’s not!<br /><br />There have also been many times when we’ve tried following up on work where we are pretty sure there has been a lot of selective reporting of positive results. Let’s just say that that is an unpleasant realization to make.<br /><br />I want to emphasize that I don’t think that people are being malicious or fraudulent in their work. I think the vast majority of scientists are honest people and are not trying to do something wrong. I just think that science would benefit from having a more transparent reporting of results, because it is sometimes the data that doesn’t fit the narrative that leads to something new in the future. I also don’t necessarily think we need to formally pre-register our work, although it might be an interesting experiment to try. We should just try and shift our culture a bit towards transparent reporting. One potential challenge in doing science this way is that our stories are a lot less likely to be “perfect”. There will almost always be some bits of conflicting evidence, and given our adversarial peer review system, there is seemingly a lot of pressure to keep these conflicting results out. Or is there? We have been doing this for quite a while, and I would say that our experience has been largely fine in the sense that reviewers don’t mind as long as you are transparent about it. I say “largely” because there have definitely been cases in which reviewers point out some issue that we were transparent about and reject our paper because of it. So at least in my experience, I would say that adopting this more transparent reporting of results is not entirely without consequence. All I can say is that if we do decide to make this cultural shift, we also have to be more tolerant of imperfections in the “story” when we put our reviewer hats on.<br /><br />By the way, I think a lot of people tend to think of selective reporting as a problem of experimental science. Not at all the case! Same goes for every analysis of e.g. some large scale dataset: if you checked for some signal in the data, you have to report the result, regardless of whether the result came out the way you wanted. It’s actually if anything even more of an issue in computational work in some ways, where many hypotheses can be tested with the same data in (relatively) rapid fashion.<br /><br />There is also a bit of a gray area in terms of what to do about false leads. Sometimes, you have an idea that goes in a new direction that has nothing to do with the story of the paper. I don’t know what to do in this case. Certainly, science would be in some ways better for having these results out there, since there was probably (hopefully?) some basis for the experiment or analysis in the first place. But it may just serve to distract from the main thread of the paper, making it harder to follow. I don’t know how best to balance these competing and important principles, but I think it’s an important discussion for us to have.<br /><br />I’m very curious how people will respond to this discussion. Ultimately, there is no form or checklist that can solve the issues we have in science. Pre-registration sounds like a bureaucratic solution, but in the end, it’s just a call for careful, honest thought about the work we do. I’m sure some people reading this will have a strongly negative reaction, much like I did at first. All I’m saying is “Sure, I hear you, just think about it.” 🙂<br />ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com1tag:blogger.com,1999:blog-5506135718533366764.post-35778524608473156862023-09-26T05:24:00.002-04:002023-09-26T05:24:16.020-04:00“Refusing the call” and presenting a scientific story<p> <span style="font-family: Arial, sans-serif; font-size: 11pt; white-space-collapse: preserve;">When scientists present in an informal setting where questions are expected, I always have an internal bet with myself as to how long until some daring person asks the first question, after which everyone else joins in and the questions rapidly start pouring out. This usually happens around the 10 minute mark. This phenomenon has gotten me wondering what this means for how best to structure a scientific talk.</span></p><span id="docs-internal-guid-fba0b556-7fff-8752-2d78-efc06c836baf"><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;">I think this “dam breaking” phenomenon can be best thought of in terms of “refusal of the call”, which is a critical part of the classic hero’s journey in the theory of storytelling. The hero typically is leading some sort of hum-drum existence, until suddenly there is a “call to adventure”. Think Luke Skywalker in Star Wars (Episode IV, of course) when </span><a href="https://www.youtube.com/watch?v=tpJnMVKO6Vo" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Obi Wan proposes that he go on an adventure to save the galaxy, only for Luke to say “Awww, I hate the empire, but what can </span><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">I</span><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;"> do about it?”</span></a><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;">. (Related point, Mark Hamill sucks.) Usually, shortly afterwards, the hero will “refuse the call” to adventure—usually from fear or lack of confidence or perhaps just from having common sense. This refusal involves some sort of rejection of the premise of the proposed adventure, which then needs to be overcome.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;">I think that’s exactly what’s going on in a scientific talk. As Nancy Duarte says, in a presentation, your audience is the hero. You are Obi Wan, presenting the call to adventure (an exciting new idea). And, almost immediately afterward, your audience (the hero) is going to refuse the call, meaning they are going to challenge your premise. In the context of a scientific talk, I think that’s where you have to present some sort of data. Like, I’ve presented you with this cool idea, here’s some preliminary result that gives it some credibility. Then the hero will follow the guide a little further on the adventure.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;">The mistake I sometimes see in scientific talks is that they let this tension go on for too long. They introduce an idea and then expound on the idea for a while, not providing the relief of a bit of data as the audience is refusing the call. The danger is that the longer the audience's mind runs with their internal criticism, the more it will forever dominate their destiny. Instead, spoon feed it to them slowly. Present an idea. Within a minute, say to the audience “You may be wondering about X. Well here is Y proof.” If you are pacing at their rate of questioning, perhaps a little faster, then they will feel very satisfied.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;">For instance:</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;">“You may think drug resistance in cancer is caused by genetic mutations and selection. However, what if it is non-genetic in origin? We did sequencing and found no mutations…”</span></p><div><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div></span>ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com0tag:blogger.com,1999:blog-5506135718533366764.post-65779541528977043992021-07-16T10:30:00.004-04:002021-07-16T10:30:52.528-04:00Confusion and credentials in presenting your work<p>Just listened to a great <a href="https://www.npr.org/transcripts/1016126310" target="_blank">Planet Money episode in which Dr. Cecelia Conrad</a> describes how she dealt with some horrible racist students in her class who were essentially questioning her credentials. She got the advice from a senior professor to be <i>less</i> clear in her intro class:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-eghydL69ed0/YPGVsaDTjfI/AAAAAAAAM9M/hk2Y5fgdYMExiW6bjEjg_nVeA66FisbwwCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1744" data-original-width="1338" height="660" src="https://lh3.googleusercontent.com/-eghydL69ed0/YPGVsaDTjfI/AAAAAAAAM9M/hk2Y5fgdYMExiW6bjEjg_nVeA66FisbwwCLcBGAsYHQ/w506-h660/image.png" width="506" /></a></div><br />This snippet reminded me of some advice I got from my postdoc advisor about giving talks: "You don't want <i>everything</i> to be clear. You should have at least some part of it that is confusing." This advice has really stuck with me through the years, and I have continued to puzzle over it for a long time. Like, it should all be clear, no? I always felt like the measure of success for a presentation should on some level be a monotonically increasing function of its clarity.<p></p><p>But… for a while before the pandemic, I was doing this QR code thing to get feedback after my talks on both degree of clarity and degree of inspiration, and I have to say I feel like I noticed some slight anti-correlation: when I gave a super clear talk, it was seemingly less inspiring, but when I got lower marks for clarity, it was somehow more inspiring. Huh.</p><p>Nancy Duarte makes the point that in any presentation, the audience is the hero, and you as the presenter are more like Yoda, the sage who leads the audience on their heroic adventure. Perhaps it is not for nothing that Yoda speaks in wise-seeming syntactically mixed-up babble. Perhaps you have to assert credentials and intellectual dominance at some point in order to inspire your audience? Thoughts on how best to accomplish that goal?</p>ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com1tag:blogger.com,1999:blog-5506135718533366764.post-88491191484750017602020-07-31T09:17:00.008-04:002020-08-03T08:47:51.699-04:00Alternative hypotheses and the Gautham TransformAs I have mentioned several times, having Gautham in the lab really changed how I think about science. In particular, I learned a lot about how to take a more critical approach to science. I think this has made me a far better and more rigorous scientist, and I want to impart those lessons to all members of the lab.<br /><br />The most important thing I learned from Gautham was to consider alternative hypotheses. I know this sounds like duh, that’s what I learn in my RCR meetings, “expected outcomes and potential pitfalls” sections of grants, and boring classes on how to do science, but I think that’s because we so rarely see how powerful it is in practice. I think it was one of Gautham’s favorite pastimes, and really exemplified his scientific aesthetic (indeed, he was very well known for demonstrating some alternative hypotheses for carrier multiplication, I believe). There were many, many times Gautham proposed alternative hypotheses in our lab, and it was always illuminating. Indeed, one of the main points of his second paper from the lab was about how one could explain “fluctuations between states” by simple population dynamics without any state switching—a whole paper’s worth of alternative hypothesis!<br /><br />Why do we generally fail to consider alternative hypotheses? One reason is that it’s scary and not fun. Generally, the hypothesis you want to consider is the option that is the fun one. It is scary to contemplate the idea that something fun might turn out to be something boring. (Gautham and I used to joke that the “Gautham Transform” was taking something seemingly interesting and showing that it was actually boring.) The truth of it, though, is that most things are boring. Sure, in biology, there are a lot more surprises than in, say, physics, but there are still far fewer interesting things than are generally claimed. I think that we would all do better to come in with a stronger prior belief that most findings actually have a boring explanation, and a critical implementation of that belief is to propose alternative hypotheses. Keep in mind also that when we are trained, we typically are presented with a list of facts with no alternatives. This manner of pedagogy leaves most of us with very little appreciation for all the wrong turns that comprise science as it’s being made as opposed to the little diagrams in the textbooks.<br /><br />The other reason we fail to consider alternatives is that it’s a lot of work. It’s always going to be harder to spend as much time actively thinking of ways to show that your pet theory is incorrect, and so in my experience it’s usually more work to come up with plausible alternative hypotheses. Usually, this difficulty manifests as a proclamation of “there’s just no other way it could be!” Thing is… there’s ALWAYS an alternative hypothesis. All models are wrong. You may get to a point where you just get tired, or the alternatives seem too outlandish, but there’s always another alternative to exclude. I remember as we were wrapping up our transcriptional-scaling-with-cell-size manuscript, we got this cool result suggesting that transcription was cut in half upon DNA replication (decrease in burst frequency). I was really into this idea, and Gautham was like, that’s really weird, there must be some other explanation. I was like, I can’t think of one, and I remember him saying “Well, it’s hard, but there has to be something, what you’re proposing is really weird”. So… I spent a couple days thinking about it, and then, voila, an alternative! (The alternative was a global decrease in transcription in S-phase, which Olivia eliminated with a clever experiment measuring transcription from a late-replicating gene.) Point is, it’s hard but necessary work.<br /><br />(Note: I’m wondering about ways to actively encourage people to consider alternatives on a more regular basis. One suggestion was to stop, say, group meeting somewhere in the middle and just explicitly ask everyone to think of alternatives for a few minutes, then check in. Another option (HT Ben Emert) is to have a lab buddy who’s job is to work with you to challenge hypotheses. Anybody have other thoughts?)<br /><br />So when do you stop making alternatives? I think that’s largely a matter of taste. At some point, you have to stand by a model you propose, exclude as many plausible alternatives as you can, and then acknowledge that there are other possible explanations for what you see that you just didn’t think of. Progress continues, excluding one alternative at a time…<br />ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com1tag:blogger.com,1999:blog-5506135718533366764.post-70162225518884777472020-07-31T08:30:00.002-04:002020-07-31T08:31:10.739-04:00“Hipster” overlay journalsBeen thinking a lot about overlay journals and their implications these days. For those who don’t know, an overlay journal is sort of like a “meta-journal” in that it doesn’t formally publish its own papers. Rather, it provides links to other preprints/papers that it thinks are interesting. On some level, the idea is that the true value of a journal is to serve as a filter for what someone thinks is science worth reading so that you don’t have to read every single paper. An overlay journal provides that filter function without the need for the rest of the (costly) trappings of a journal, like peer review and, uhh, color figures ;). <div><br /></div><div>There is one very interesting aspect of an overlay journal that I don’t think has been discussed very much: in contrast with regular journals, they are fundamentally non-exclusive, meaning that ANY overlay journal can in principle “publish” ANY paper. What this non-exclusivity means is that there is no jockeying between journals to publish the “obviously important” papers, which have a perhaps slightly elevated chance of actually being important. You know, like “we sequenced 10x more single cells than the last paper in a fancy journal” kind of papers. If you run an overlay journal, you never have to gaze longingly at those “high impact” papers—if you want to publish it, just add it to your overlay!</div><div><br /></div><div>What are the consequences of non-exclusivity? Primarily, I think it would serve to diminish the value of “obviously important” papers. Everyone can identify them based on authors and number of genomes sequenced or whatever, so there’s really not that much value in including them <i>per se</i>. It would be like saying “Here’s my playlist, it’s like a copy of the Billboard Top 40”. Nobody’s going to look to your overlay journal for that kind of stuff (which you can readily get from CNS or Twitter). Rather, the real value would be in making lists of papers that are awesome but might otherwise be overlooked—essentially a hipster playlist. As an editor, your cache would be in your ability to identify these new, cool papers and making Michael Cera-esque mixtapes out of them. Can leave the Hot 100 to Casey Kasem/Spotify algorithms.</div><div><br /></div><div>Measuring the importance of an overlay journal would also be interesting. Clearly, impact factor is not a useful metric, since anybody can make their impact factor as high as they want by including highly cited papers. I would guess a far more sensible metric would be number of followers of the journal (which makes more sense anyway).</div><div><br /></div><div>Another interesting aspect of an overlay journal is that it can be retrospective. You could include old papers as well, highlighting old gems that may have been forgotten.</div><div><br /></div><div>Of course, an interesting question is whether there is any difference between an overlay journal and someone’s Twitter feed. Not sure, actually…<br /></div><div><br /></div><div>Also, thoughts on existing journals that have hipster qualities to them? I vote Current Biology, my lab votes eLife.</div>ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com2tag:blogger.com,1999:blog-5506135718533366764.post-30001472829915787032020-07-17T13:57:00.000-04:002020-07-17T13:57:14.105-04:00My favorite "high yield" guides to telling better stories<span id="docs-internal-guid-bb5883e1-7fff-b975-15ea-b2cf3c7385f0"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><i>Guest post by Eric Sanford</i></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">In medical school, we usually have five lectures’ worth of new material to memorize each day. Since we can’t simply remember it all, we are always seeking “high yield” resources (a term used so often by med students that it quickly becomes a joke): those concise one or two-pagers that somehow contain 95 percent of what we need to know for our exams. My quest of finding the highest yield resources has continued in full force after becoming a PhD student.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">A major goal of mine has been to improve my scientific communication skills (you know, writing, public speaking, figure-making… i.e. those extremely-important skills that most of us scientists are pretty bad at), and I’ve come across a few very high yield resources as I’ve worked on this. Here are my favorites so far:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Resonate</span><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">, by Nancy Duarte:</span></p><ul style="margin-bottom: 0px; margin-top: 0px;"><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The best talks are inspiring, but “be more inspiring” is not easy advice to follow.</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This book teaches you how to turn your content into a story that inspires an audience.</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I received </span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">extremely positive</span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> feedback and a lot of audience questions the first time I gave a talk where I tried to follow the suggestions of this book.</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This was both the most fun and the most useful of all my recommendations.</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">The Visual Display of Quantitative Information</span><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">, by Edward Tufte:</span></p><ul style="margin-bottom: 0px; margin-top: 0px;"><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Tufte is probably the most famous “data visualization” guru, and I think this book, his first book, is his best one. (I’ve flipped through the sequels and would also recommend the chapter on color from “Envisioning Information.”)</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This book provides a useful framework for designing graphics that convey information in ways that are easy (easier?) for readers to understand. Some pointers include removing clutter, repeating designs in “small multiples”, labeling important elements directly, and using space consistently when composing multiple elements in the same figure.</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">The Elements of Style</span><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">, by Strunk and White, pages 18-25:</span></p><ul style="margin-bottom: 0px; margin-top: 0px;"><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Available at </span><a href="http://www.jlakes.org/ch/web/The-elements-of-style.pdf" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">http://www.jlakes.org/ch/web/The-elements-of-style.pdf</span></a></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Pages 18-25 are especially high yield and provide concrete advice (with examples) on how to make stylistic changes that make writing both more efficient, specific, and enjoyable to read.</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Words to Avoid When Writing, </span><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">by Arjun Raj</span></p><ul style="margin-bottom: 0px; margin-top: 0px;"><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Available at </span><a href="https://docs.google.com/document/d/1r6nDcF43esu3xBjmk3ERAmaEHKEB75_HflSkk3zZhBk/edit" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">https://docs.google.com/document/d/1r6nDcF43esu3xBjmk3ERAmaEHKEB75_HflSkk3zZhBk/edit</span></a></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Turns out this blog’s creator has learned a few things about science writing!</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This is an important list to review when writing a grant or publication: each of the words, while commonly used, are a sign of sloppy thinking.</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Each word has a worked example for how to replace it with something better.</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Raj Lab basic Adobe Illustrator (CC) guide</span><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">, by Connie Jiang</span></p><ul style="margin-bottom: 0px; margin-top: 0px;"><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Available at </span><a href="https://docs.google.com/document/d/1TXmbltzBPcApCcuJ9HLOIQgWPqKylrFRWRudrN-5vBE/edit#heading=h.or1to9c1y8il" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">https://docs.google.com/document/d/1TXmbltzBPcApCcuJ9HLOIQgWPqKylrFRWRudrN-5vBE/edit#heading=h.or1to9c1y8il</span></a></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">If you have access to Illustrator, this is a fantastic resource for making or improving scientific figures.</span></p></li><li dir="ltr" style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Worth reading each page, but also a great reference for specific problems or questions.</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">There are many other great resources out there that are also worth going through if you have the time (</span><span style="font-family: arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Style: Lessons in Clarity and Grace</span><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> by Bizup and Williams is another excellent writing guide), but for me these ones above had the highest amount-learned-per-minute-of-concentration-invested. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><i>Guest post by Eric Sanford</i></span></p><div><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><i><br /></i></span></div><div><span style="font-family: arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div></span>Eric Sanfordhttp://www.blogger.com/profile/15961577098475463213noreply@blogger.com3tag:blogger.com,1999:blog-5506135718533366764.post-73282654653035427562019-08-21T22:44:00.000-04:002019-08-21T22:44:57.471-04:00I <3 Adobe Illustrator (for scientific figure-making) and I hope that you will too<div dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;">
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;"><i>Guest post by Connie Jiang</i></span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;">As has been covered somewhat extensively (see <a href="https://rajlaboratory.blogspot.com/2016/01/a-proposal-for-how-to-label-small.html">here</a>, <a href="https://rajlaboratory.blogspot.com/2016/02/from-reproducibility-to-over.html">here</a>,</span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;"> </span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;">and <a href="https://rajlaboratory.blogspot.com/2017/08/figure-scripting-and-how-we-organize.html">here</a></span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;">), we are a lab that really appreciates the flexibility and ease with which one can use Illustrator to compile and annotate hard-coded graphical data elements to create figures. Using Illustrator to set things like font size, marker color, and line weighting is often far more intuitive and time-efficient than trying to do so programmatically. Furthermore, it can easily re-arrange/re-align graphics and create beautiful vector schematics, with far more flexibility than hard-coded options or PowerPoint.</span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">So why don’t more people use Illustrator?</span><br />
<span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span></span>
<span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">For one, it’s not cheap. We are lucky to have access to relatively inexpensive licenses through Penn. If expense is your issue, I’ve heard good things about Inkscape and Gimp, but unfortunately I have minimal experience with these and this document will not discuss them. Furthermore, as powerful and flexible as Illustrator is, its interface can be overwhelming. Faced with the activation energy and cognitive burden of having to learn how to do even basic things (drawing an arrow, placing and reshaping a text box without distorting the text it contains), maybe it’s unsurprising that so many people continue to use PowerPoint, a piece of software that most people in our lab first began experimenting with prior to 8th grade [AR editor’s note: uhhh… not everyone]. </span></span><br />
<span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span></span>
<span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Recently, I decided to try to compile a doc with the express purpose of decreasing that activation energy of learning to use Illustrator to accomplish tasks that we do in the lab setting. Feel free to skip to the bottom if you’d just like to get to that link, but here were the main goals of this document:</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<ol>
<li><span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Compile a checklist to run through for each figure before submission. This is a set of guidelines and standards we aim to adhere to in lab to maintain quality and consistency of figures.</span></span></li>
<li><span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Give a basic but thorough rundown of essentially everything in Illustrator that you need to begin to construct a scientific figure. Furthermore, impart the Illustrator “lingo” necessary to empower people to search for more specific queries.</span></span></li>
<li><span style="white-space: pre-wrap;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Answer some of what I feel to be the most FAQs. Due to my love of science-art and general artistic/design experimentation, I’ve spent a lot of time in Illustrator, so people in lab will sometimes come to me with questions. These are questions like: “my figure has too many points and is slowing my Illustrator down: how can I fix it?” and “what’s the difference between linked and embedded images?”. Additionally, there are cool features that I feel like every scientist should be able to take advantage of, like “why are layers super awesome?” and “how can I select everything of similar appearance attributes?”.</span></span></li>
</ol>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Finally, a disclaimer: This document will (hopefully) give you the tools and language to use Illustrator as you see fit. It does not give any design guidance or impart aesthetic sense (aside from heavily encouraging you to not use Myriad Pro). Make good judgments~</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><span style="font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Full Raj lab basic Illustrator guide can be found <a href="https://docs.google.com/document/d/1TXmbltzBPcApCcuJ9HLOIQgWPqKylrFRWRudrN-5vBE/edit#">here</a>.</span></span></div>
</div>
Connie Jianghttp://www.blogger.com/profile/07414227520258683189noreply@blogger.com1tag:blogger.com,1999:blog-5506135718533366764.post-29880971828725044732019-08-04T13:33:00.000-04:002019-08-04T13:33:18.671-04:00I need a coachI’ve been ruminating over the course of the last several years on a conversation I had with Rob Phillips about coaches. He was saying (and hopefully he will forgive me if I’m mischaracterizing this) that he has had people serve the role of coach in his life before, and that that really helped push him to do better. It’s something I keep coming back to over and over, especially as I get further along in my career.<br />
<br />
In processing what Rob was saying, one of the first questions that needed answering is exactly what is a coach? I think most of us think about formal training interactions (i.e., students, postdocs) when we think of coaching in science, and I think this ends up conflating two actually rather disparate things, which are mentoring and coaching. At least for me, mentorship is about wisdom that I have accumulated about decision making that I can hopefully pass on to others. These can be things like “Hmm, I think that experiment is unlikely to be informative” or “That area of research is pretty promising” or “I don’t think that will matter much for a job application, I would spend your time on this instead”. A coach, on the other hand, is someone who will help push you to focus and implement strategies for things you already know, but are having trouble doing. Like “I think we can get this experiment done faster” or “This code could be more cleanly written” or “This experiment is sloppy, let’s clean it up”. Basically, a mentor gives advice on what to do, a coach gives advice on how to actually do it.<br />
<br />
Why does this decoupling matter, especially later in your career? When in a formal training situation, you will often get both of these from the same people—the same person, say, guiding your research project is the same person pushing you to get things done right. But after a few years in a faculty position, the N starts to get pretty small, and as such I think the value of mentorship per se diminishes significantly; basically, everybody gives you a bunch of conflicting advice on what to do in any given situation, which is frankly mostly just a collection of well-meaning but at best mildly useful anecdotes. But while the utility of mentorship decreases (or perhaps the availability of high quality mentorship) decreases, I have found that I still have a need for someone to hold me accountable, to help me implement the wisdom that I have accumulated but am sometimes too lazy or scared to put into practice. Like, someone to say “hey, watch a recording of your lecture finally and implement the changes” or “push yourself to think more mechanistically, your ideas are weak” or “that writing is lazy, do better” or “finish that half-written blog post”. To some extent, you can get this from various people in your life, and I desperately seek those people out, but it’s increasingly hard to find the further along you are. Moreover, even if you do find someone, they may have a different set of wisdom that they would be trying to implement for you, like, coaching you towards what they think is good, not what you yourself think is good (“Always need a hypothesis in each specific aim” whereas maybe you’ve come to the conclusion that that’s not important or whatever). If you have gotten to the point where you’ve developed your own set of models of what matters or doesn’t in the world, then you somehow need to be able to coach yourself in order to achieve those goals.<br />
<br />
Is it possible to self-coach? I think so, but I’ve always struggled to figure out how. I guess the first step is to think about what makes a good coach. To me, the role of a good coach is to devise a concrete plan (often with some sort of measurable outcome) that promotes a desired change in default behavior. For example, when working with people in the lab in a coaching capacity, one thing I’ve tried to do is to propose concrete goals to try and help overcome barriers. If someone could be participating more in group meeting and seminars, I’ll say “try to ask at least 3 questions at group meeting and one at every seminar” and that does seem to help. Or I’ll push someone to make their figures, or write down their experiment along with results and conclusions. Or make a list of things to do in a day and then search for one more thing to add. Setting these sorts of rules can help provide the structure to achieve these goals and model new behaviors.<br />
<br />
How do you implement these coaching strategies for yourself? I think there are a few steps, the first of which are relatively easy. Initially, the issue is to identify the issue, which is actually usually fairly clear: “I want to reduce time spent on email”, “I want to write clean code”, “I want to construct a set of alternative hypotheses every time I come up with some fun new idea”, “Push myself to really think in a model-based fashion”. Next, is reduction to a concrete set of goals, which is also usually pretty easy: “Read every email only once and batch process them for a set period of time” or “write software that follows XYZ design pattern” or “write down alternative hypotheses”. The biggest struggle is accountability, which is where having a coach would be good. How do I enforce the rules when I’m the only one following them?<br />
<br />
I’m not really sure, but one thing that works for me (which is perhaps quite obvious) is to rely on something external for accountability. For example, I am always looking for ways to improve my talks, and value being able to do a good job. However, it was hard to get feedback, and even when I did, I often didn’t follow through to implement said feedback. So I did this thing where I show the audience a QR code which leads them to a form for feedback. Often, they pointed out things I didn’t realize were unclear, which was of course helpful. But what was also helpful was when they pointed out things that I already knew were unclear, but had been lazy about fixing. This provided me with a bit of motivation to finally fix the issue, and I think it’s improved things overall. Another externalization strategy I’ve tried is to imagine that I’m trying to model behavior for someone else. Example: I was writing some software a while back for the lab, and there were times where I could have done something in the quick, lazy, and wrong way, rather than in the right way. What helped motivate me to do it right was to say to myself, “Hey, people in the lab are going to look at this software as an example of how to do things, and I need to make sure they learn the right things, so do it right, dummy”.<br />
<br />
Some things are really hard to externalize, like making sure you stress test your ideas with alternative hypotheses and designing the experiments that will rigorously test them. One form of externalization that works for me is to imagine former lab members who were really smart and critical and just imagine them saying to me “but what about…”. Just imagining what they might say somehow helps me push myself to think a bit harder.<br />
<br />
Any thoughts on other ways to hold yourself accountable when nobody else is looking?ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com0tag:blogger.com,1999:blog-5506135718533366764.post-53047215464944165672019-05-06T21:22:00.000-04:002019-05-06T21:22:05.062-04:00Wisdom of crowds and open, asynchronous peer reviewI am very much in favor of preprints and open review, but <a href="https://www.npr.org/templates/transcript/transcript.php?storyId=430372183" target="_blank">something I listened to on Planet Money</a> recently gave me some food for thought, along with a recent poll I tweeted about re-reviewing papers. The episode was about wisdom of the crowds, and how magically if you take a large number of non-expert guesses about, say, the weight of an ox, the average comes out pretty close to the actual value. Pretty cool effect!<br />
<br />
But something in the podcast caught my ear. They talked about how when they asked some kids, you had to watch out, because once one kid said, say, 300 pounds (wildly inaccurate), then <i>if the other kids heard it</i>, then they would all start saying 300 pounds. Maybe some minor variations, but the point is that they were strongly influenced by that initial guess, rather than just picking something essentially purely random. The thing was that <i>if you had no point of reference, then even a guess provides that point of reference.</i><br />
<br />
Okay, so what does this have to do with peer review? What got me thinking about it was the tweet about re-reviewing a paper you had already seen but for a different journal. I'm like nah not gonna do it because it's a waste of time, but some people said, well, you are now biased. So… in a world where we openly and asynchronously review papers (preprints, postpub, whatever), we would have the same problem that the kids guessing the weight of the cow did: whoever gives the first opinion would potentially strongly influence all subsequent opinions. With conventional peer review, everyone does it blind to the others, and so reviews could be considered more independent samplings (probably dramatically undersampled, but that's another blog post). But imagine someone comments on a preprint with some purported flaw. That narrative is very likely to color subsequent reviews and discussions. I think we've all seen this coloring: take eLife collaborative peer review, or even grant review. Everyone harmonizes their scores, and it's often not an averaging. One could argue that unlike randos on the internet guessing a cow's weight, peer reviewers are all experts. Maybe, but I am somehow not so sure that once we are in the world of experts reviewing what is hopefully a reasonably decent paper that there's much signal beyond noise.<br />
<br />
What could we do about this? Well, we could commission someone to hold all the open reviews in confidence and then publish them all at once… oh wait, I think we already have some annoying system for that. I dunno, not really sure, but anyway, was something I was wondering about recently, thoughts welcome.ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com2tag:blogger.com,1999:blog-5506135718533366764.post-49846469100197808032019-04-28T11:29:00.001-04:002019-04-28T11:32:53.122-04:00Reintegrating into lab following a mental health leave<b>[From AR]</b> These days, there is a greatly increased awareness and decreased stigmatization of mental health amongst trainees (and faculty, for that matter), which is great. For mentors, understanding mental health issues amongst trainees is super important, and something we have until recently not gotten a lot of training on. More recently, it is increasingly common to get some training or at least information on how to recognize the onset of mental health issues, and in graduate groups here at Penn at least, it is fairly straightforward to initiative a leave of absence to deal with the issue, should that be required. However, one aspect of handling mental health leaves for which there appears to be precious little guidance out there is what challenges trainees face when returning from a mental health leave of absence, and what mentors might do about it. Here, I present a document written by four anonymous trainees with some of their thoughts (and I will chime in at the end with some thoughts from the mentor perspective).<br />
<div>
<br /></div>
<div>
<span id="docs-internal-guid-8b55671d-7fff-13e8-7766-a30debd878c4"></span><br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span id="docs-internal-guid-8b55671d-7fff-13e8-7766-a30debd878c4"><span style="font-family: "arial"; font-size: 11pt; font-style: italic; vertical-align: baseline; white-space: pre-wrap;"><b>[From trainees]</b> This article is a collection of viewpoints from four trainees on mental health in academia. We list a collection of helpful practices on the part of the PI and the lab environment in general for cases when the trainees return to lab after recovering from mental health issues. </span></span></blockquote>
<span id="docs-internal-guid-8b55671d-7fff-13e8-7766-a30debd878c4">
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">A trainee typically returns either because they feel recovered and ready to get back to normalcy, or they are **better** than before and have self-imposed goals (e.g. finishing their PhD), or they just miss doing science. Trainees in these situations are likely to have spent time introspecting on multiple fronts and they often return with renewed drive. However, it is very difficult to shake off the fear of recurrence of the episode (h</span><span style="font-family: "arial"; font-size: 11pt; white-space: pre-wrap;">ere we use episode broadly to refer to a phase of very poor mental health)</span><span style="font-family: "arial"; font-size: 11pt; white-space: pre-wrap;">, which can make trainees more vulnerable and sensitive to external circumstances than an average person; for instance, minor stresses can appear much larger. In particular, an off-day post a mental health issue can make one think they are already slipping back into it. In some cases, students may find it more difficult to start a new task, perhaps due to the latent fear of not being able to learn afresh. Support from the mentor and lab environment in general can be crucial in both providing and sustaining the confidence of the trainee. </span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">It is important that the mentor recognize that the act of returning to the lab is an act of courage in itself.</span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> The PI’s interactions with the trainee have a huge bearing on how the trainee re-integrates into his/her work. Here are some steps that we think can help:</span></blockquote>
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Explicitly tell trainees to seek the PI out if they need help.</span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> This can be important for all trainees to hear because the default assumption is that these are personal problems to be dealt with personally in its entirety. In fact, advisors should do this with every trainee -- explicitly tell them that they are there to be reached out to, should their mental health be compromised/affected in any way. Restating this to a returning trainee can help create a welcoming and safe environment.</span></blockquote>
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Reintegrating the trainee into the lab environment.</span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> The PI should have an open conversation with the trainee about how much information they want divulged to the rest of the group/department, and how they communicate the trainee’s absence to the group, if at all.</span></blockquote>
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Increased time with the mentee. </span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">More frequent meetings with a returning student for the first few months help immensely for multiple reasons: a. It can help quell internal fears by a process of regular reinforcement; b. It can get the students back on track with their research faster; c. The academically stimulating conversations can provide the gradual push needed to think at a level they were used to before mental health issues. Having said that, individuals have their preferred way of dealing with the re-entering situation and a frank conversation about how they want to proceed helps immensely. </span></blockquote>
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Help rebuild the trainee’s confidence. </span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">One of the authors of this post recounts her experience of getting back on her feet. Her advisor unequivocally told her: “Your PhD will get done; you are smart enough. You just need to work on your mental health, and I will work with you to make that the first priority.” Words of encouragement can go a long way -- there is ample anecdotal evidence that people can fully recover from their mental health state if proper care is taken by all stakeholders. </span></blockquote>
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Create a small, well-defined goal/team goals. </span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">One of the authors of this article spent her first few months working on a fairly easy and straightforward project with a clear message, one that was easy to keep pushing on as she settled in to lab again. While this may not be the best way forward for everyone depending on where they are with their research, a clearly-defined goal can come as a quick side-project, or a deliberate breaking-down of a large project into very actionable smaller ones. Another alternative is to allow the trainee to work with another student/postdoc, something which allows a constant back-and-forth, and quicker validation which can lead to less room for mental doubt.</span></blockquote>
<br />
<blockquote class="tr_bq" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"></span><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline;">Remember that trainees may need to come back for a variety of other reasons as well.</span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline;"> There are costs associated with a prolonged leave of absence, and for some trainees, they may need to come back before they are totally done with their mental health work. It's likely that some time needs to be set aside to continue that work, and it's helpful if PIs can work with students to accommodate that, within reason.</span></span><span style="font-family: "arial";"><span style="font-size: 14.6667px; white-space: pre-wrap;"><br /></span></span><br />
<span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">Finally, it is important for all involved parties to realize that the job of a PI is not to be the trainee’s parent, but to help the student along in their professional journey. Facilitating a lab environment where one feels comfortable, respected, and heard goes a long way, even if that means going an extra mile on the PI’s part to ensure such conditions, case-by-case.</span></blockquote>
</span><div>
<span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<b>[Back to AR]</b> Hopefully this article is helpful for mentors and also for trainees as they try to reintegrate into the lab. For my part as a mentor, I think that a little extra empathy and attention can go a long way. I think it's important for all parties to realize that mentors are typically not trained mental health professionals, but some common sense guidelines could include increased communication, reasonable expectations, and in particular a realization that tasks that would seem quite easy for a trainee to accomplish before might be much harder now at first, in particular anything out of the usual comfort zone, like a new technique, etc.</div>
<div>
<br /></div>
<div>
Comments more than welcome; it seems this is a relatively under-reported area. And a huge thank you to the anonymous writers of this letter for starting the discussion.<br />
<div>
<span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com4tag:blogger.com,1999:blog-5506135718533366764.post-14250951091174391382019-02-18T22:30:00.001-05:002019-02-18T22:30:25.381-05:00Dear me, I am awesome. Sincerely, me… aka How to write a letter of rec for yourselfGot an email from someone who got asked to write a letter for themselves by someone else and was looking for guidance… haha, now that PI has made work for me! :) Oh well, no problem, I actually realize how hard this is for the letter drafter, and it’s also something for which there is very little guidance out there for obvious reasons. So I thought I’d make a little guide. Oh, first a couple things. First off, I don’t really know all that much about doing this, having written a few for myself and having asked for a couple, so comments from others are most welcome. Secondly, if you’re one of those sanctimonious types who thinks the PIs should write every letter and never ask for a draft, well, this blog post is probably not for you so don’t bug me about it. Third, if the PI is European, maybe just like turn everything down a notch, ya know? ;)<br /><br />Anyhoo: so I figure the best way to describe how to do this is to describe how I write a letter. I’ll aim it at how I write letters for, say, a former trainee applying for a postdoc fellowship, maybe with some notes about how this might change for faculty applying for some sort of award or something.<br /><br />Okay. I usually use the first paragraph to give an executive summary. Here’s an example of what I might write:<br /><blockquote class="tr_bq">
“It is my pleasure to provide my strongest possible recommendation for Dr. Nancy Longpaper. Nancy is simply an incredible scientist: she has developed, from scratch and by combining both experimental and computational skills, a system that has led to fundamental new insights into the evolution of frog legs. She has all the tools to be a superstar in her field: talent, intellectual brilliance, work ethic, and raw passion for science to become a stellar independent scientist. I look forward to watching her career unfold in the coming years.”</blockquote>
<div>
Or whatever something like that. The key parts of this that you will want to leave blank is the first sentence, i.e., the “strongest possible recommendation” part. That’s an important part that the letter writer will fill in.<br /><br />Okay, second (optional) paragraph. This one depends a bit on personality. For some letter writers, they like to include a bit about how awesome they are and thus how qualified they are to write the letter. This is important for things like visas and so forth. This could be something like “First, I would like to introduce myself and my expertise. My laboratory studies XYZ, and I am an expert in ABC. I have published several peer reviewed articles in renowned journals such as Proceedings of the Canadian Horticultural Society B and our work has been continuously funded by the NIH.” I personally don’t include things like this for regular (non-visa) recommendations, but I have seen it.<br /><br />Third paragraph: I usually try and put in some context about how I met the person I’m recommending. Like, “I first met Nancy when she was looking for labs to rotate in. She rotated in my lab and worked on project ABC. Even in her short time in the lab, she managed to accomplish XYZ. I immediately offered her a spot, and while I was disappointed for her to join Prof. Goodgrant’s lab, I was very pleased when she asked me to chair her thesis committee.” If you are a junior PI, this might be replaced with something about how the letter writer knows about your work and any interactions you may have had.<br /><br />Next several paragraphs: a bunch of scientific meat. This is where you are REALLY going to save your letter writer some time. I usually break it into two parts. First paragraph or two, I describe the person’s work. What specifically did they do. PROVIDE CITATIONS, including journal names. Sorry, they matter, too bad. Try and aim for a very general audience, stressing primarily the impact of the findings. But if you don’t, don’t worry, people probably either know the work already or not. Still, try. Emphasize specific contributions. Like, “Nancy herself conceived of the critical set of controls that was required to establish the now well accepted ‘left leg bias estimator’ statistical methodology that was the key to making the discovery that XYZ.” At all times, emphasize why what you did was special. Don’t be shy! If you’re too ridiculous, don’t worry, your letter writer will fix it.<br /><br />Next part of the science-meat section: in my letters, I usually try and zoom out a bit. Like, what are the specific attributes of the person that led them to be successful in the aforementioned science. Like, “This is a set of findings that only someone of Nancy’s caliber could have discovered. Her intellectual abilities and broad command of the literature enabled her to rapidly ask important questions at the forefront of the field…” Be careful to emphasize big picture important qualities and not just list out your specific skills here. Like, don’t say “Nancy was really good at qPCR and probably ran about 4.32 million of them.” Makes you sound like a drone. At the trainee level, something about how rapidly you picked up skills could be good, but definitely not at the junior faculty level. Just try and be honest about the qualities you have that you think are most important and relevant. Be maybe a little over the top but not too crazy and then maybe your letter writer will embellish as needed.<br /><br />Second to last paragraph: I try and fill in a bit more personal characteristics here. Like, what are the personal qualities that helped them shine. E.g. “Nancy also is an excellent communicator of her science, and already has excellent visibility. She gives great talks and has generated a lot of enthusiasm……” Also, if relevant, can add the standard “On a personal note, Nancy is a wonderful person to have in the lab……” Probably like 4-5 sentences max. Make it sound like you belong at the level you are applying for. If it’s for a faculty position, make it sound like you are faculty, not a student.<br /><br />Finally, I end my letters with an “In sum, Nancy is the perfect candidate for XYZ. I have had the privilege of watching many star scientists develop into independent scientists in this field at top institutions over the years, and I consider Nancy to be of that caliber. I cannot recommend her more strongly.” This one can be sort of a skeleton and the letter writer can fill this in with whatever gushy verbiage they want. For some things, there might be some sort of “comparables” statement here that they can put in if they want.<br /><br />Tips:</div>
<div>
<ul>
<li>Don’t ever say anything bad. If you say something bad, it’s a huge red flag. If the letter writer wants to say something bad, they will. That would be a pretty jerky thing to do, though.</li>
<li>Length: There are three things that matter in a letter: the first paragraph, the last paragraph, and how long the letter is in between. For a postdoc thingy, aim for 1.5-2 pages for a strong letter. 2-3 for faculty positions. 1-2 for other stuff after that.</li>
<li>Duplication: What do you do if two letter writers ask for a draft? Uhhhh… not actually sure. I have tried to make a few edits, but sometimes I just send it and say hey already sent this and they can kinda edit it up a bit. I dunno, weird situation.</li>
</ul>
Anyway, that’s my template for whatever it’s worth, and comments welcome from anyone who knows more!</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com0tag:blogger.com,1999:blog-5506135718533366764.post-78217356961417681562019-02-03T17:09:00.002-05:002019-02-03T17:09:39.102-05:00The sad state of scientific talks (and a thought on how we might help fix it)Just got back from a Keystone meeting, and I’m just going to say it (rather than subtweet it): most of the talks were bad. I don’t mean to offend anyone, and certainly it was no worse than most other conferences, but come on. Talks over time, filled with jargon and unexplained data incomprehensible to those even slightly outside the field, long rambling introductions… it doesn’t have to be this way, people! Honestly, it also begs the question as to why people bother going to these meetings just to play around on their computers because the talk quality is so poor. I’ve heard so many people say the informal interactions are the most useful thing at conferences. I actually think this is partly because the formal part is so bad.<br />
<br />
Why? After all, there are endless resources out there on how to give a good talk. While some tips conflict (titles? no titles? titles? no titles?), mostly they agree on some basic tenets of slide construction and presentation. I wrote <a href="http://rajlaboratory.blogspot.com/2016/09/some-thoughts-on-how-to-structure-talk.html" target="_blank">this blog post</a> with some tips on structuring talks and also links to a few other resources I think are good. And most graduate programs have at least some sort of workshop or something or other on giving a talk. So why are we in this situation?<br />
<br />
I think the key thing to realize is that giving a good talk actually requires working on your talk. A good talk requires more than taking a couple minutes to throw some raw data onto a slide and winging it with how you present that data. For most of us, when we write a paper, it is a long iterative process to achieve clarity and engagement. Why would a talk be any different? (Oh, and by the way, practice is critical, but is not in and of itself sufficient—have to work on the right things; see aforementioned blog post.)<br />
<br />
I think the fundamental issue is the nature of feedback and incentives for giving research talks. Without having these structured well, there is little push to do the work required to make a talk good, and they are currently structured very poorly. For incentives, the biggest problem is that the structure to date is all about what you don’t get in the long term, which are often things you don’t know you could get in the first place. Giving a good talk has huge benefits and opens the door to various opportunities long term, but it’s not like someone is going to tell you, “Hey, I had this job opening, but I’m not going to tell you about it now because your talk stunk." Partly, the issue is that the visible benefits of good presentations are often correlated to some extent with brilliance. Take, for instance, Michael Elowitz’s talk at this conference, which my lab hands down voted as the best talk of the conference. Amazing science, clear, and exciting. Michael is a brilliant and deservedly highly successful scientist. Does it help that he is an excellent communicator of his work? Of course! To what extent? I don’t know. What I can say is that many of the best scientists presented their work very well. Where do cause and effect begin and end? Hard to say, but it’s clearly not an independent variable.<br />
<br />
Despite this correlation, I still firmly believe that you don’t have to Michael Elowitz-level brilliant to give a great talk. So then why are all these talks so bad? The other element beyond vague incentives is feedback. The most common feedback, regardless of anything about the talk you give, is “Hey, great talk!” Maybe, if you really stunk it up, you’ll get “interesting talk”. And that’s about it. I have many times gotten “Hey, great talk” followed by a question demonstrating that I totally did a terrible job explaining things. I mean, how is anybody ever going to get better if they don’t even get a thumbs-up/down on their presentation? The reason we don’t get that feedback is obviously because of the social awkwardness to telling someone something they did publicly was bad. The main place where people feel safe to give feedback is in lab meeting, which while somewhat helpful is also one of the worst places to get feedback. Asking a bunch of people already intimately familiar with your story and conversant in your jargon about what is clear or not is not going to get you all that far, generally. Also, the person with the most authority in that context (the PI) probably also gives terrible talks and so is not a good person to get feedback from. (Indeed, I have heard many, many stories of PIs actively giving their trainees bad advice.) Generally, the fact that most people you are getting feedback from aren’t themselves typically good at it is a big problem.<br />
<br />
Okay, fine…<br />
<br />
WHAT CAN WE DO ABOUT IT?<br />
<br />
<div>
Again, I think the key missing element is honest feedback—I think most talk-givers don’t even realize just how bad their talks are. As I said, few people are going to tell someone to their face that their talk sucks. So how about the following: what if people preregister their talk on a website, and then people can anonymously submit a rating with comments? Basically like a teacher rating, but for speakers at a conference. You could even provide the link to the rating website on the first slide of your talk or something. This would have a number of advantages. First off, if you don’t want to do it, fine, no problem. Second, all feedback is anonymous, thus allowing people to be honest. Also, the comments allow people to give some more detailed feedback if they so choose. And, there is a strong positive incentive. With permission, you could have your average rating posted. This rating could be compared to e.g. the overall average, and if it’s good—which presumably it is if you decided to share it :)—then that’s great publicity, no?<br />
<br />
One problem with this, though, is it doesn’t necessarily provide specific feedback. Like, what was clear or not? Comments could provide this to some extent. Also, if you, as the speaker, are willing, you could even imagine posting some questions related to your talk and seeing how well people got those particular points. Of course completely optional and just for those who really care about improving. Which should be all of us, right? :)<br />
<br />
Oh, and one suggestion from Rita Strack was to promote the 15 minute format, which is short enough to either require concision and clarity, or, should that not happen, is over fast! :)</div>
<div>
<br /></div>
<div>
Some suggested (e.g. Katie Whitehead) that we incentivize good talks by doing Skype interviews or having them submit YouTubes, etc. for contributed talks. In principle I like this, but I think it's just a LOT of work and also conflates scientific merit with presentation merit, so people who don't get a spot have something other than their presentation skills to blame. Still, could work maybe.<br />
<br />
Another, perhaps more radical idea, is to do away with the talk format entirely. Most scientists are far more clear when answering questions (probably for the simple reason that the audience drives it). Perhaps we could limit talks to 5 minutes followed by some sort of structured Q&A? Not sure how to do that exactly, but anyway, a thought.<br />
<br />
Anybody want to give this a try?</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com5tag:blogger.com,1999:blog-5506135718533366764.post-41657004549737627752018-08-08T16:50:00.001-04:002018-08-09T04:00:14.682-04:00On mechanism and systems biology(Latest in a slowly unfolding series of blog posts from the Paros conference.)<br />
<br />
<b>Related reading:</b><br />
<br />
<ul>
<li>Musings on Mechanism, Rob Phillips, <a href="https://www.ncbi.nlm.nih.gov/pubmed/28963318">https://www.ncbi.nlm.nih.gov/pubmed/28963318</a></li>
<li>Excellent blog post on "Theoretical Amnesia" <a href="http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/">http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/</a>)</li>
</ul>
<br />
<span id="docs-internal-guid-dd5a9d8e-7fff-9569-050a-434e0e1f961c"><span style="font-family: "arial"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="296" src="https://lh4.googleusercontent.com/gW2qkUHbaz4CmjS9yDpyEc2kNq0oNHWh_psyL7t4JdBxma90zlnGHEv-uflvalgeN98iFGBdBLZUiTaBIb2IS79sKi-1mZkqmj8skDG1Ox5lGZgPRGCkRxQ5x30KHs75VXfq9itr" style="border: none; transform: rotate(0rad);" width="624" /></span></span><br />
<br />
Mechanism. The word fills many of us with dread: “Not enough mechanism.” “Not particularly mechanistic.” "What's the mechanism?" So then what exactly do we mean by mechanism? I don’t think it’s an idle question—rather, I think it gets down to the very essence of what we think science means. And I think there are some practical consequences on everything from how we report results to the questions we may choose to study (and consequently to how we evaluate science). So I’ll try and organize this post around a few concrete proposals.<br />
<br />
To start: I think the definition I’ve settled on for mechanism is “a model for how something works”. <br />
<br />
I think it’s interesting to think about how the term mechanism has evolved in our field from something that really was mechanism once upon a time into something that is really not mechanism. In the old days, mechanism meant figuring out e.g. what an enzyme did and how it worked, perhaps in conjunction with other enzymes. Things like DNA polymerase and ATP synthase. The power of the hard mechanistic knowledge of this era is hard to overstate.<br />
<br />
What can we learn about the power of mechanism/models from this example?<br />
<br />
As the author of <a href="http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/">this post</a> argues, models/theories are “inference tickets” that allow you to make hard predictions in completely new situations without testing them. We are used to thinking of models as being written in math and making quantitative predictions, but this need not be the case. Here, the predictions of how these enzymes function has led to, amongst other things, our entire molecular biology toolkit: add this enzyme, it will phosphorylate your DNA, add this other enzyme, it will ligate that to another piece of DNA. That these enzymes perform certain functions is a “mechanism” that we used to predict what would happen if we put these molecules in a test tube together, and that largely bore out, with huge practical implications.<br />
<br />
Mechanisms necessarily come with a layer of abstraction. Perhaps we are more used to talking about these in models, where we have a name for them: “assumptions”. Essentially, there is a point at which we say, who knows, we’re just going to say that this is the way it is, and then build our model from there. In this case, it’s that the enzyme does what we say it will. We still have quite a limited ability to take an unknown sequence of amino acids and predict what it will do, and certainly very limited ability to take a desired function and just write out the sequence to accomplish said function. We just say, okay, assume these molecules do XYZ, and then our model is that they are important for e.g. transcription, or reverse transcription, or DNA replication, or whatever.<br />
<br />
Fast forward to today, when a lot of us are studying biological regulation, and we have a very different notion of what constitutes “mechanism”. Now, it’s like oh, I see a correlation between X and Y, the reviewer asks for “mechanism”, so you knock down X and see less Y, and that’s “mechanism”. Not to completely discount this—I mean, we’ve learned a fair amount by doing these sorts of experiments, but I think it’s a pretty clear that this is not sufficient to say that we know how it works. Rather, this is a devolution to empiricism, which is something I think we need to fix in our field.<br />
<br />
Perhaps the most salient question is what it does it mean to know “how it works?”. I posit that mechanism is an inference that connects one bit of empiricism to another. Let’s illustrate in the case of something where we do know the mechanism/model: a lever.<br />
<br />
<img height="176" src="https://lh3.googleusercontent.com/zDX2rVFyaAW4oPfENcOuuQCBmB6Ryuvl0cVcLXZXkGGgL-dkGNDG_YgaSy0ZT_MjBiP69B6zV5sL1boAcuMRXmZb9Eyk2nmHoDo9X0ZNwywWqIfb9ggkT3e3HeqVfFLnpEetxhww" width="320" /><br />
<br />
<br />
<br />
<br />
“How it works” in this context means that we need a layer of abstraction, and have some degree of inference given that layer of abstraction. Here, the question may be “how hard do I have to push to lift the weight?”. Do we need to know that the matter is composed of quarks to make this prediction, or how hard the lever itself is? No. Do we need to know how the string works? No. We just assume the weight pulls down on the string and whatever it’s made of is irrelevant because we know these to be empirically the case. We are going to assume that the only things that matter are the locations of the weight, the fulcrum, and my finger, as well as the weight of the, uhh, weight and how hard I push. This is the layer of abstraction the model is based on. The model we use is that of force balance, and we can use that to predict exactly how hard to push given these distances and weights.<br />
<br />
How would a modern data scientist approach this problem? Probably take like 10,000 levers and discover Archimedes Law of the Lever by making a lot of plots in R. Who knows, maybe this is basically how Archimedes figured it out in the first place. It is perhaps often possible to figure out a relationship empirically, and even make some predictions. But that’s not what we (or at least I) consider a mechanism. I think there has to be something beyond pure empiricism, often linking very disparate scales or processes, sometimes in ways that are simply impossible to investigate empirically. In this case, we can use the concepts of force to figure out how things might work with, say, multiple weights, or systems of weights on levers, or even things that don’t look like levers at all. Wow!<br />
<br />
<div>
Okay, so back to regulatory biology. I think one issue that we suffer from is that what we call mechanism has moved away from true “how it works” models and settled into what is really empiricism, sort of without us noticing it. Consider, for instance, development. People will say, oh, this transcription factor controls intestinal development. Why do they say that? Well, knock it out and there’s no intestine. Put it somewhere else and now you get extra intestine. Okay, but that’s not how it works. It’s empirical. How can you spot empiricism? A good sign is excessive obsession with statistics: effect sizes and p-values are often a good sign that you didn’t really figure out how it works. Another sign is that we aren’t really able to apply what we learned outside of the original context. If I gave you a DNA typewriter and said, okay, make an intestine, you would have no idea how to do it, right? We can make more intestine <i>in the original context</i>, but the domain of applicability is pretty limited.<br />
<br />
Personally, I think that these difficulties arise partially because of our tools, but mostly because I think we are still focused on the wrong layers of abstraction. Probably the most common current layers of abstraction are those of genes/molecules, cells, and organisms. Our most powerful models/mechanisms to date are the ones where we could draw straight lines connecting these up. Like, mutate this gene, make these cells look funny, now this person has this disease. However, I think these straight lines are more the exception than the norm. Mostly, I think these mappings are highly convoluted in interwoven systems, making it very hard to make predictions based on empiricism alone (future blog post coming on Omnigenic Model to discuss this further).<br />
<br />
Which leads me to a proposal: let’s start thinking about other layers of abstraction. I think that the successes of the genes/molecules -> cells paradigm has led to a certain ossification of thought centered around thinking of genes and molecules and cells as being the right layers of abstraction. But maybe genes and cells are not such fundamental units as we think they are. In the context of multicellular organisms, perhaps cells themselves are passive players, and rather it is communities of cells that are the fundamental unit. Organoids could be a good example of this, dunno. Also, it is becoming clear that genetics has some pretty serious limits in terms of determining mechanism in the sense I’ve defined. Is there some other layer involving perhaps groups of genes? Sorry, not a particularly inspired idea, but whatever, something like that maybe. Part of thinking this way also means that we have to reconsider how we evaluate science. As <a href="https://www.ncbi.nlm.nih.gov/pubmed/28963318">Rob pointed out</a>, we have gotten so used to equating “mechanism” to “molecules and their effects on cells” that we have become both closed minded to other potential types of mechanism while also deceiving ourselves into allowing empiricism to pose as mechanism under the guise of statistics. We just have to be open to new abstractions and not hold everyone to the "What's the molecule?" standard.<br />
<br />
Of course, underlying this is an open question: do such layers of abstraction that allow mechanism in the true sense exist? Complexity seems to be everywhere in biology, and my reaction so far has been to just throw up my hands up and say “it’s complicated!”. But (and this is another lesson learned from Rob), that’s not an excuse—we have to at least try. And I do think we can find some mechanistic wormholes through the seemingly infinite space of empiricism that we are currently mired in.<br />
<br />
Regardless of what layers of abstraction we choose, however, I think that it is clear that a common feature of these future models will be that they are multifactorial, meaning that they will simultaneously incorporate the interactions of multiple molecules or cells or whatever the units we choose are. How do we deal with multiple interactions? I’m not alone in thinking that our models need to be quantitative, which as noted in my first post, is an idea that’s been around for some time now. However, I think that a fair charge is that in the early days of this field, our quantitative models were pretty much window dressing. I think (again a point that I’ve finally absorbed from Rob) that we have to start setting (and reporting) <i>quantitative</i> goals. We can’t pick and choose how our science is quantitative. If we have some pretty model for something, we better do the hard work to get the parameters we need, make hard quantitative predictions, and then stick to them. And if we don’t quantitatively get what we predict, we have to admit we were wrong. Not partly right, which is what we do now. Here’s the current playbook for a SysBio paper: quantitatively measure some phenomenon, make a nice model, predict that removal of factor X should send factor Y up by 4x, measure that it went up 2x, and put a bow on it and call it a day. I think we just have to admit that this is not good enough. This “pick and choose” mix of quantitative and qualitative analyses is hugely damaging because it makes it impossible to build upon these models. The problem is that qualitative reporting in, say, abstracts leads to people seeing “X affects Y” and “Y affects Z” and concluding “thus, X affects Z” even though the effects for X on Y and Y on Z may be small enough to make this conclusion pretty tenuous.<br />
<br />
So I have a couple proposals. One is that in abstracts, every statement should include some sort of measure of the percentage of effect explained by the putative mechanism. I.e., you can’t just say “X affects Y”. You have to say something like “X explains 40% of the change in Y”. I know, this is hard to do, and requires thought about exactly what “explains” means. But yeah, science is hard work. Until we are honest about this, we’re always going to be “quantitative” biologists instead of true quantitative biologists.<br />
<br />
Also, as a related grand challenge, I think it would be cool to try and be able to explain some regulatory process in biology out to 99.9%. As in, okay, we really now understand in some pretty solid way how something works. Like, we actually have mechanism in the true sense. You can argue that this number is arbitrary, and it is, but I think it could function well as an aspirational goal.<br />
<br />
Any discussion of empiricism vs. theory will touch on the question of science vs. engineering. I would argue that—because we’re in an age of empiricism—most of what we’re doing in biology right now is probably best called engineering. Trying to make cells divide faster or turn into this cell or kill that other cell. And it’s true that look, whatever, if I can fix your heart, who cares if I have a theory of heart? One of my favorite stories along these lines is the story of how fracking was discovered, which was purely by accident (<a href="https://www.npr.org/2016/09/27/495671385/how-an-engineers-desperate-experiment-created-fracking">see Planet Money podcast</a>): a desperate gas engineer looking to cut costs just kept cutting out an expensive chemical and seeing better yield until he just went with pure water and, voila, more gas than ever. Why? Who cares! Then again, think about how many mechanistic models went into, e.g., the design of the drills, transportation, everything else that goes into delivering energy. I think this highlights the fact that just like science and engineering are intertwined, so are mechanism and empiricism. Perhaps it’s time, though, to reconsider what we mean by mechanism to make it both more expansive and rigorous.</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com8tag:blogger.com,1999:blog-5506135718533366764.post-14813575437687380972018-08-06T11:38:00.001-04:002018-08-06T11:38:27.696-04:00The biologist's arrow<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]-->
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false"
DefSemiHidden="false" DefQFormat="false" DefPriority="99"
LatentStyleCount="375">
<w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 9"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="header"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footer"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index heading"/>
<w:LsdException Locked="false" Priority="35" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of figures"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope return"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="line number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="page number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of authorities"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="macro"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="toa heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 5"/>
<w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Closing"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Signature"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Message Header"/>
<w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Salutation"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Date"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Block Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="FollowedHyperlink"/>
<w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Document Map"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Plain Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="E-mail Signature"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Top of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Bottom of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal (Web)"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Acronym"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Cite"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Code"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Definition"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Keyboard"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Preformatted"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Sample"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Typewriter"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Variable"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Table"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation subject"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="No List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Contemporary"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Elegant"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Professional"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Balloon Text"/>
<w:LsdException Locked="false" Priority="39" Name="Table Grid"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Theme"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" QFormat="true"
Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" QFormat="true"
Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" QFormat="true"
Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" QFormat="true"
Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" QFormat="true"
Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" QFormat="true"
Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" SemiHidden="true"
UnhideWhenUsed="true" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/>
<w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/>
<w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/>
<w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/>
<w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/>
<w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/>
<w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/>
<w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hashtag"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Unresolved Mention"/>
</w:LatentStyles>
</xml><![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0in;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;
mso-header-margin:.5in;
mso-footer-margin:.5in;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
</style>
<![endif]-->
<!--StartFragment-->
<br />
<div class="MsoNormal" style="line-height: 150%;">
<i style="mso-bidi-font-style: normal;"><span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">Guest post by Caroline Bartman<o:p></o:p></span></i><br />
<i style="mso-bidi-font-style: normal;"><span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;"><br /></span></i></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">How do we understand biology? “Mutant IDH2 <i style="mso-bidi-font-style: normal;"><arrow> </i>2-hydroxyglutarate <i style="mso-bidi-font-style: normal;"><arrow></i> hypermethylation <i style="mso-bidi-font-style: normal;"><arrow>
</i>cell proliferation (?),” I scribbled at the top of a paper I read this week.
My mind requires linear relationships, direct chains of cause and effect, to
retain the findings of a paper I read.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">Evidence suggests that this is not how biology in general operates. For
example, Pritchard’s ‘omnigenic theory’ synthesizes many years of work to show
that most polymorphisms contribute to the total phenotype in a significant but barely
detectable way. Identifying each genetic variant that contributes to a
phenotype requires many years of costly effort and will culminate with a long
list of polymorphisms that incrementally contribute to a phenotype. (Exceptions
to this rule- PCSK9- are valuable but rare.) Not only are most contributions
miniscule (median contribution of significant height SNPs is 0.00143 meters
according to Pritchard), but many polymorphisms play a role in a wide range of
traits, by influencing broadly expressed genes. Our search for cause <i style="mso-bidi-font-style: normal;"><arrow> </i>effect reveals a tangled
thicket of partial causes and modest effects.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">Human genetic studies are not the only realm where such complexity
dominates. We perform RNA sequencing of wild-type and knockout cells, find a
thousand differentially expressed genes, and then focus on a single target
gene. We do a screen and follow up on a single hit. It boggles the mind to
understand that all of the hits, probably even some below the significance
threshold, contribute to that biological process every time it occurs. So we
ignore this tangle in order to tell a story, to write a paper, to give a talk
that other scientists will appreciate.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">This struggle to understand continues as we try to finish a study. Many
scientific projects reach an uncomfortable stage where we have a phenotype in
hand, a dramatic finding with some relevance to an open biological question, but
we require a bit of mechanism for the last figure. (We use the phrase ‘bit of
mechanism’ with a half-ashamed laugh.) A bit of mechanism? A handle to give
readers, to reassure them that biology is not random, there is a reason for our
finding, there is ultimately something to understand? How many of these last
figure gambits are quickly abandoned by the relevant subfield as future studies
fail to support these ‘mechanisms,’ or change their interpretation beyond
recognition? <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">How do we as humans with limited intelligence, limited bandwidth,
limited attention span understand complex biological processes? <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">Does understanding biology even matter? Don’t we do biology to help
patients, to solve problems, to cure disease? But one of the most attractive
things about biology for me was that there is a truth outside oneself. Unlike
consulting, or writing, or reporting, which are all ways humans can talk about
humans, or operate in artificial systems constructed by humans, I believed that
science was the way to escape from navel-gazing, the way out of the closed loop.
It is not all about humans and feelings and opinions! There are truths outside
our selves that we can understand! Just look at ribosomes, or whales, or frogs,
or the lac operon and you see a truth that does not require humans as an origin
but that humans could find a logic behind. But can we actually understand that
logic? <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc; font-family: "arial" , sans-serif; font-size: 11.0pt; line-height: 150%;">This concern does not lend itself well to selecting and starting a new
biological project. The papers that are most beautiful and elegant to me are
the simplest. But they leave me with a disquieting feeling that they have
achieved beauty by denying complexity. <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="color: #cccccc;"><br /></span></div>
<!--EndFragment--><br />Caroline Bartmanhttp://www.blogger.com/profile/04420722703020468795noreply@blogger.com1tag:blogger.com,1999:blog-5506135718533366764.post-84942696114464734772018-06-14T08:57:00.000-04:002018-06-17T06:42:33.129-04:00Notes from Frontiers in Biophysics conference in Paros, episode 1 (pilot): Where's the beef in biophysics?Long blog post hiatus, which is a story for another time. For now, I’m reporting from what was a very small conference on the Frontiers of Biophysics from Paros, a Greek island in the Aegean, organized by Steve Quake and Rob Phillips. The goals of the conference were two-fold:<br />
<div>
<ol>
<li>Identify big picture goals and issues in biophysics, and</li>
<li>Consider ways to alleviate suffering and further human health.</li>
</ol>
Regarding the latter, I should say at the outset that this conference was very generously supported by Steve through the foundation he has established in memory of his mother-in-law Eleftheria Peiou, who sounds like she was a wonderful woman, and suffered through various discomforts in the medical system, which was the inspiration behind trying to reduce human suffering. I actually found this directive quite inspiring, and I’ve personally been wondering what I could do in that vein in my lab. I also wonder whether the time is right for a series of small Manhattan Projects on various topics so identified. But perhaps I’ll leave that for a later post.<br />
<br />
Anyway, it was a VERY interesting meeting in general, and so I think I’m going to split this discussion up based on themes across a couple different blog posts, probably over the course of the next week or two. Here are some topics I’ll write about:<br />
<br />
Exactly what is all this cell type stuff about<br />
<br />
Exactly what do we mean by mechanism<br />
<br />
I need a coach<br />
<br />
What are some Manhattan Projects in biology/medicine<br />
<br />
Maybe some others<br />
<br />
So the conference started with everyone introducing themselves and their interests (research and otherwise) in a 5 minute lightning talk, time strictly enforced. First off, can I just say, what a thoughtful group of folks! It is clear that everyone came prepared to think outside their own narrow interests, which is very refreshing.<br />
<br />
The next thing I noticed a lot of was a lot of hand-wringing about what exactly we mean by biophysics, which is what I’ll talk about for the rest of this blog post. (Please keep in mind that this is very much an opinionated take and does not necessarily reflect that of the conferees.) To me, basically, biophysics, as seemingly defined at this meeting, as a whole needs a pretty fundamental rebranding. Raise your hand if biophysics means one of the following to you:<br />
<ol>
<li>Lipid rafts</li>
<li>Ion channels</li>
<li>A bunch of old dudes trying to convince each other how smart they are (sorry, cheap shot intended for all physicists) ;)</li>
</ol>
If you have not raised your hand yet, then perhaps you’re one of the lonely self-proclaimed “systems biologists” out there, a largely self-identified group that has become very scattered since around 2000. What is the history of this group of people? Here’s a brief (and probably offensive, sorry) view of molecular biology. Up until the 80s, maybe 90s, molecular biology had an amazing run, working out the genetic code, signaling, aspects of gene regulation, and countless other things I’m forgetting. This culminated in the “gene-jock” era in which researchers could relate a mutation to a phenotype in mechanistic detail (this is like the <a href="http://rajlaboratory.blogspot.com/2014/12/origin-and-impact-of-stories-in-life.html" target="_blank">Cell golden era I blogged about earlier</a>). Since that era, well… not so much progress, if you ask me—I’m still firmly of the opinion that there haven’t really been any big conceptual breakthroughs in 20-30 years, except Yamanaka, although one could argue whether that’s more engineering. I think this is basically the end of the one-gene-one-phenotype era. As it became clear that progress would require the consideration of multiple variables, it also became clear that a more quantitative approach would be good. For ease of storytelling, let’s put this date around 2000, when a fork in the road emerged. One path was the birth of genomics and a more model-free statistical approach to biology, one which has come to dominate a lot of the headlines now; more on that later. The other was “systems biology”, characterized by an influx of quantitative people (including many physicists) into molecular biology, with the aim of building a quantitative mechanistic model of the cell. I would say this field had its heyday from around 2000-2010 (“Hey look Ma, I put GFP on a reporter construct and put error bars on my graph and published it in Nature!”), after which folks from this group have scattered towards more genomics-type work or have moved towards more biological applications. I think that this version of "systems biology" most accurately describes most of the attendees at the meeting, many of whom came from single molecule biophysics.<br />
<br />
I viewed this meeting as a good opportunity to maybe take score and see how well our community has done. I think Steve put it pretty concisely when he said “So, where’s the beef?” I.e., it's been a while, and so what does our little systems biology corner of the world have to show for itself in the world of biology more broadly? Steve posed the question at dinner: “What are the top 10 contributions from biophysics that have made it to textbook-level biology canon?” I think we came up with two: Hodgkin and Huxley’s model of action potentials, gene expression “noise”, and Luria and Delbrück’s work on genetic heritability (and maybe kinetic proofreading; other suggestions more than welcome!). Ouch. So one big goal of the meeting was to identify where biophysics might go to actually deliver on the promise and excitement of the early 2000s. Note: Rob had a long list of examples of cool contributions, but none of them has gotten a lot of traction with biologists.<br />
<br />
I’ll report more on some specific ideas for the future later, but for now, here’s my personal take on part of the issue. With the influx of physicists came an influx of physics ideas. And I think this historical baggage mostly distracts from the problems we might try to solve (Stephan Grill made this point as well, that we need something fundamentally new ways of thinking about problems). This baggage from physics is I think a problem both strategically and tactically. At the most navel-gazy level, I feel like discussions of “Are we going to have Newton’s laws for biology” and “What is going to be the hydrogen atom of the cell” and “What level of description should we be looking at” never really went anywhere and feel utterly stale at this point. On a more practical level, one issue I see is trying to map quantitative problems that come up in biology back to solved problems in physics, like the renormalization group or Hamiltonian dynamics or what have you. Now, I’m definitely not qualified to get into the details of these constructs and their potential utility, but I can say that we’ve had physicists who are qualified for some time now, and I think I agree with Steve: where’s the beef?<br />
<br />
I think I agree with Stephan that perhaps we as a community perhaps need to take stock of what it is that we value about the physics part of biophysics and then maybe jettison the rest. To me, the things I value about physics are quantitative rigor and the level of predictive power that goes with it (more on that in blog post on mechanism). I love talking to folks who have a sense for the numbers, and can spot when an argument doesn’t make quantitative sense. Steve also mentioned something that I think is a nice way to come up with fruitful problems, which is looking at existing data through a quantitative lens to be able to find paradoxes in current qualitative thinking. To me, these are important ways in which we can contribute, and I believe will have a broader impact in the biological community (and indeed already has through the work of a number of “former” systems biologists).<br />
<br />
To me, all this raises a question that I tried to bring up at the meeting but that didn’t really gain much traction in our discussions, which is how do we define and build our community? So far, it’s been mostly defined by what it is not: well, we’re quantitative, but not genomics; we’re like regular biology, but not really; we’re… just not this and that. Personally, I think our community could benefit from a strong positive vision of what sort of science we represent. And I think we need to make this vision connect with biology. Rob made the point, which is certainly valid, that maybe we don’t need to care about what biologists think about our work. I think there’s room for that, but I feel like building a movement would require more than us just engaging in our own curiosities.<br />
<br />
Which of course begs the question of why we would need to have a “movement” anyway. I think there’s a few lessons to learn from our genomics colleagues, who I think have done a much better job of creating a movement. I think there are two main benefits. One is attracting talent to the field and building a “school of thought”. The other is attracting funding and so forth. Genomics has done both of these extremely well. There are dangers as well. Sometimes genomics folks sound more like advocates than scientists, and it’s important to keep science grounded in data. Still, overall, I think there are huge benefits. Currently, our field is a bunch of little fiefdoms, and like it or not, building things bigger than any one person involves a political dimension.<br />
<br />
So how do we define this field? One theme of the conference that came up repeatedly was the idea of Hilbert Problems, which for those who don’t know, is a list of open math problems set out in 1900 by David Hilbert, and they were very influential. Can we perhaps build a field around a set of grand challenges? I find that idea very appealing. Although I think that given that I’ve increasingly come to think of biology as engineering instead of science, I wonder if maybe phrasing these questions instead in engineering terms would be better, sort of like a bunch of biomedical Manhattan Projects. I’ll talk about some ideas we came up with in a later blog post.<br />
<br />
Anyway, more in the coming days/weeks…</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com9tag:blogger.com,1999:blog-5506135718533366764.post-21858390184755538372017-10-04T21:13:00.001-04:002017-10-04T21:13:35.331-04:00How to train a postdoc? - by Uschi Symmons<span style="color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">- by Uschi Symmons</span></span><br />
<span style="color: white;"><br /></span>
<br />
<div dir="ltr" id="docs-internal-guid-7c5c6711-e894-b75a-3711-090477cebdc3" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">A couple of weeks ago I was roped into a twitter discussion about postdoc training, which seemed to rapidly develop into a stalemate between the parties: postdocs, who felt they weren't getting the support and training they wanted and needed, and PIs, who felt their often substantial efforts were being ignored. Many of the arguments sounded familiar: over the past two years I’ve been actively involved in our postdoc community, and have found that when it comes to postdocs, often every side feels misunderstood. This can lead to a real impasse for improvements, so in this blog post I’ve put together a couple of points summarizing problems and some efforts we've made to work around these to improve training and support. </span></span></div>
<span style="color: white;"><br /></span>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><u><b><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; vertical-align: baseline;">First off, here some of the problems we encountered:</span></b></u></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><b><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">1. postdocs are a difficult group to cater for, because they are a </span></b><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline;">very diverse group</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> in almost every aspect: </span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">- work/lab experience and goals</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: ranging from college-into-grad-school-straight-into-postdoc to people who have multi-year work experience outside academia to scientists who might be on their second or third postdoc. This diversity typically also translates into future ambitions: many wish to continue in academic research, but industry/teaching/consulting/science communication are also part of the repertoire.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">- training</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: Some postdocs come from colleges and grad schools with ample opportunity for soft-skill training. Others might never have had a formal course in even such trivial things, like paper writing or how to give a talk.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">- postdoc duration</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: there is a fair amount of variation in how long postdocs stay, depending on both personality and field of research. In our department postdocs, for example, postdoc positions vary widely, ranging from 1-2 years (eg computational sciences, chemistry) to 5-7 years (biomedical sciences).</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">- nationality</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: I don’t know if postdocs are actually more internationally diverse than grad students, but the implications of that diversity are often greater. Some postdocs might be preparing for a career in the current country, others might want to return to their home country, which makes it difficult to offer them the same kind of support. Some postdocs may have stayed in the same country for a long time and know the funding system inside-out, others may have moved country repeatedly and have only a vague idea about grant opportunities.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">- family status</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: when I was in grad school three people in my year (<5%) had kids. In our postdoc group that percentage is way higher (I don’t have numbers, but would put it around 30-40%), and many more are in serious long-term relationships, some of which require long commutes (</span><a href="https://tenureshewrote.wordpress.com/2013/09/18/solving-the-two-body-problem/" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">think two-body problem</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">). Thus, organising postdoc events means dealing with people on very diverse schedules.</span></span></div>
<span style="color: white;"><br /></span>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">2. In addition postdocs are also often </span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline;">a smaller group</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> than grad students. For example, at UPenn, we have as many postdocs in the School of Engineering as we have grad students in a single department of the school (Bioengineering). If fact, I have often heard disappointed faculty argue that postdocs “don’t make use of available resources”, because of low turnout at events. In my experience this is not the case: organising as a grad student and a postdoc I have found that turnout is typically around 30-40% - postdoc events simply seem less attended, because the base is so much smaller.</span></span></div>
<span style="color: white;"><br /></span>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">3. Finally, Postdocs frequently have </span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline;">lower visibility</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: whereas grad students are typically seen by many faculty during the recruitment process or during classes, it is not unusual for postdocs to encounter only their immediate working group. And unlike grad students, postdocs do not come in as part of a cohort, but at different times during the year, making it also difficult to plan things like orientation meetings, where postdocs are introduced to the department in a timely manner.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> </span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">Seeing all of the above, it is a no-brainer why training postdocs can be difficult. On one hand problems are conceptual: Do you try to cater to everyone’s needs or just the majority? Do you try to help the “weakest link” (the people with least prior training) or advance people who are already at the front of the field? On the other hand, there are also plenty of practical issues: Do you adjust events to the term calendar, even if postdocs arrive and leave at different times? Do you organise the same events annually or every couple of years? Is it OK to have evening/weekend events?</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> But these are not unsolvable dilemmas. <u><b>Based on our experiences during the past two years, here are some practical suggestions*</b></u>:</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<ol id="docs-internal-guid-7c5c6711-e8a3-9ebf-cb4c-252c6a80833a" style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">Pool resources/training opportunities</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> with the grad school and/or other postdoc programmes close-by: for a single small postdoc program, it is impossible to cater to all needs. But more cross-talk between programs means more ground can be covered. Such cross-talk is most likely going to be a win-win situation, both because it bolsters participant numbers and because postdocs can contribute with their diverse experiences (eg in a “how to write a paper” seminar; even postdocs who want more formal training will have written at least one paper). Our postdoc programme certainly benefits from access to the events from </span><a href="http://www.med.upenn.edu/postdoc/" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">UPenn’s Biomedical Programme</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">, as well as a growing collaboration with </span><a href="https://bmes.seas.upenn.edu/gabe/" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">GABE</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">, our department’s graduate association.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
</li>
<li dir="ltr" style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">Have a well(!)-written, up-to-date wiki/resource page AND make sure you tell incoming postdocs about this</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">. As a postdoc looking for information about pretty much anything (taxes, health insurance, funding opportunities) I often feel like Arthur in the <i>Hitchhiker’s Guide to the Galaxy</i>: </span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br class="kix-line-break" /></span><a href="http://hitchhikersguidequotes.tumblr.com/post/14333727462/mr-prosser-but-mr-dent-the-plans-have-been"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><img height="394" src="https://lh4.googleusercontent.com/oej9CBl5znubBjiGNHOzeJuPazbwrtA4QqwOKmLtnANeYeZvn4MZEnl-61a75p29Q74mNEZmzCR1LK8IvoLzrtJNvtsqrDIFqypiZT2KWaOx_UJ2YUzVyR2oIIXFfrMJCMJvRtqV" style="border: medium none; transform: rotate(0rad);" width="400" /></span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br class="kix-line-break" /></span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br class="kix-line-break" /></span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">Once you know where to look and what you’re looking for, it can be easy to find, but occasionally I am completely blindsided by things I should have known. This can be especially problematic for foreign postdocs (I’ve written </span><a href="https://pasteursquadrant.wordpress.com/2015/12/28/scientists-abroad-citizens-of-the-world-or-second-class-citizens/" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">more about that here</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">), and so telling postdocs ahead of time about resources can avoid a lot of frustration. A good time for this could be when the offer letter is sent or when postdocs deal with their initial admin. Our department still doesn’t have a streamlined process for this, but I often get personal enquiries, and I typically refer postdocs to either the National Postdoc Association's <a href="http://www.nationalpostdoc.org/default.asp?page=SurvivalGuide">Survival Guide</a> for more general advice or the aforementioned Biomedical Postdoc Program for more UPenn-related information<a href="https://www.blogger.com/null">.</a></span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
</li>
<li dir="ltr" style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">Have an open dialogue with postdocs and listen to their needs:</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> More often than not, I encounter PIs and admin </span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">who want to help </span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">postdocs. They provide training in areas they have identified as problematic, and given the diversity of the postdoc group most likely that training is genuinely needed by some. But often postdocs would like more: more diversity, other types of training or maybe they even completely different pressing issues. Yet, without open dialogue between departmental organisers and the postdoc community it’s hard to find out about these needs and wishes. Frustratingly, one tactic I encounter frequently is departmental organisers justifying the continuation or repetition of an event based on it's success, without ever asking the people who did not attend, or wondering if a different event would be equally well received. To build a good postdoc program, universities and departments need to get better at gauging needs and interests, even if this might mean re-thinking some events, or how current events are integrated into a bigger framework.</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br class="kix-line-break" /></span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">This can be difficult. As a case in point, Arjun, my PI, likes to point out that, when asked, the vast majority of postdocs request training in how to get a faculty position. So departments organise events about getting faculty positions. In fact, I am swamped with opportunities to attend panel discussions on “How to get a job in academia”: we have an annual one in our School, multiple other departments at the university host such discussions and it’s a much-favored trainee event at conferences. But after seeing two or three such panels, there’s little additional information to be gained. This does not mean that departments should do away with such panels, but coordinating with other departments (see point 1) or mixing it up with other events (eg by rotating events in two to three year cycles) would provide the opportunity to cater to the additional interests of postdocs. </span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br class="kix-line-break" /></span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">Frequent topics I’ve heard postdocs ask for are management skills, teaching skills, grant writing and external feedback/mentoring by faculty. For us, successful new programs included participation in a </span><a href="http://www.itmat.upenn.edu/juniorinvestigatorsymposium.html" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">Junior Investigators Symposium</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> on campus, which included two most positively received sessions about writing K/R awards and a “speed mentoring” session, where faculty provided career feedback in a 10-minute, one-on-one setting. Similarly, postdocs at our school who are interested in teaching can partake in training opportunities by UPenn’s </span><a href="https://www.ctl.upenn.edu/programs-and-services-post-docs-penn" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">Center for Teaching and Learning</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">, and those interested in industry and the business side of science can make use of </span><a href="https://www.pci.upenn.edu/pci-fellows/" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">a paid internship program by Penn’s Center for Innovation</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> to learn about IP and commercialization. While only a small number of postdocs make use of these opportunities per year, the provide a very valuable complement to the programs offered by the school/department. </span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br /></span></span></div>
</li>
<li dir="ltr" style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">Make a little bit of money go a long way</span><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">: Many fledgling postdoc programs, such as ours, operate on a shoestring. Obviously, in an ideal world neither PIs nor administrative bodies should shy away from spending money on postdoc training - after all, postdocs are hired as trainees. But in reality it is often difficult to get substantial monetary support: individual PIs might not want to pay for events that are not of interest for their own postdocs (and not every event will cater for every postdoc) and admin may not see the return on investment for activities not directly related to research. However, you may have noticed that many of the above suggestions involved little or no additional financial resources: faculty are often more than willing to donate their time to postdoc events, postdocs themselves can contribute to resources such as wikis, and collaborations with other programs on campus can help cover smaller costs. In addition, individual postdocs may have grants or fellowships with money earmarked for training. Encouraging them to use those resources can be of great value, especially if they are willing to share some of the knowledge they gained. My EMBO postdoctoral fellowship paid for </span><a href="http://lab-management.embo.org/course-overview" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">an amazing 3-day lab management course</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">, and I am currently discussing with our graduate association to implement some of the training exercises that we were taught. </span></span></div>
</li>
</ol>
<span style="color: white;"><br /></span>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">As my final point I’d like to say that I personally very rarely encounter faculty who consider postdocs cheap labor. If anything, most PIs I talk to have their postdocs best interest at heart. Similarly, postdocs are often more than willing to organize events and mediate the needs of their fellows. However, in the long run the efforts of individual PIs and postdocs cannot replace a well-organized institutional program, which I think likely will require taking on board some of my above suggestions and building them into a more systematic training program.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">----------------------------------------------------------------</span></span></div>
<span style="color: white;"><br /><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">*The National Postdoc Association has a much more elaborate toolkit for </span><a href="https://npamembers.site-ym.com/?page=PDA_toolkit" style="text-decoration: none;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline;">setting up and maintaining a postdoc association</span></a><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"> and there's also <a href="http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005664">a great article about initiating and maintaining a postdoc organisation</a> by Bruckman and Sebestyen. However, not all postdoc groups have the manpower or momentum to directly dive into such an program, so the tips listed here are more to get postdocs involved initially and create that sense of community and momentum to build an association.</span></span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<span style="color: white;"><span style="background-color: transparent; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"></span></span>Anonymoushttp://www.blogger.com/profile/05178742117360404630noreply@blogger.com2tag:blogger.com,1999:blog-5506135718533366764.post-19052564588650008882017-08-02T21:11:00.000-04:002017-08-02T22:09:54.470-04:00Figure scripting and how we organize computational work in the labSaw a recent <a href="https://twitter.com/casey6r0wn/status/892190846602313728" target="_blank">Twitter poll from Casey Brown</a> on the topic of figure scripting vs. "Illustrator magic", the former of which is the practice of writing a program to completely generate the figure vs. putting figures into Illustrator to make things look the way you like. Some folks really like programming it all, while I've <a href="http://rajlaboratory.blogspot.com/2016/02/from-reproducibility-to-over.html" target="_blank">argued</a> that I don't think this is very efficient, and so arguments go back on forth on Twitter about it. Thing is, I think ALL of us having this discussion here are already way in the right hand tail in terms of trying to be tidy about our computational work, while many (most?) folks out there haven't ever really thought about this at all and could potentially benefit from a discussion of what an organized computational analysis would look like in practice. So anyway, here's what we do, along with some discussion of why and what the tradeoffs are (including talking about figure scripting.<br />
<br />
First off, what is the goal? Here, I'm talking about how one might organize a computational analysis in finalized form for a paper (will touch on exploratory analysis later). In my mind, the goal is to have a well-organized, well-documented, readable and, most importantly, complete and consistent record of the computational analysis, from raw data to plots. This has a number of benefits: 1. it is more likely to be free of mistakes; 2. it is easier for others (including within the lab) to understand and reproduce the details of your analysis; 3. it is more likely to be free of mistakes. Did I mention more likely to be free of mistakes? Will talk about that more in a coming post, but that's been the driving force for me as the analyses that we do in the lab become more and more complex.<br />
<br />
[If you want to skip the details and get more to the principles behind them, please skip down a bit.]<br />
<br />
Okay, so what we've settled on in lab is to have a folder structured like this (version controlled or Dropboxed, whatever):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-ts6ExfGB7yo/WYJn4uXxX8I/AAAAAAAAJwo/E2A3c8QFHJohAT0-3Z0X1ZfZBuApma7UwCLcBGAs/s1600/Screen%2BShot%2B2017-08-02%2Bat%2B8.01.12%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="303" data-original-width="348" height="173" src="https://4.bp.blogspot.com/-ts6ExfGB7yo/WYJn4uXxX8I/AAAAAAAAJwo/E2A3c8QFHJohAT0-3Z0X1ZfZBuApma7UwCLcBGAs/s200/Screen%2BShot%2B2017-08-02%2Bat%2B8.01.12%2BPM.png" width="200" /></a></div>
<br />
<br />
I'll focus on the "paper" folder, which is ultimately what most people care about. The first thing is "extractionScripts". This contains scripts that pull out numbers from data and store them for further plot-making. Let me take this through the example of image data in the lab. We have a large software toolset called rajlabimagetools that we use for analyzing raw data (and that has it's own whole set of design choices for reproducibility, but that's a story for another day). That stores, alongside the raw data, analysis files that contain things like spot counts and cell outlines and thresholds and so forth. The extraction scripts pull data from those analysis files and puts it into .csv files, which are stored in extractedData. For an analogy with sequencing, this is like maybe taking some form of RNA-seq data and setting up a table of TPM values in a .csv file. Or whatever, you get the point. plotScripts then contains all the actual plotting scripts. These load the .csv files and run whatever to make graphical elements (like a series of histograms or whatever) and stores them in the graphs folder. finalFigures then contains the Illustrator files in which we compile the individual graphs into figures. Along with each figure (like Fig1.ai), we have a Fig1readme.txt that describes exactly what .eps or .pdf files from the graphs folders ended up in, say, Figure 1f (and, ideally, what script). Thus, everything is traceable back from the figure all the way to raw data. Note: within the extractionScripts is a file called "extractAll.m" and in plotScripts "plotAll.R" or something like that. These master scripts basically pull all the data and make all the graphs, and we rerun these completely from scratch right before submission to make sure nothing changed. Incidentally, of course, each of the folders often has a massive number of subfolders and so forth, but you get the idea.<br />
<br />
What are the tradeoffs that led us to this workflow? First off, why did we separate things out this way? Back when I was a postdoc (yes, I've been doing various forms of this since 2007 or so), I tried to just arrange things by having a folder per figure. This seemed logical at the time, and has the benefit that the output of the scripts are in close proximity to the script itself (and the figure), but the problem was that figures kept getting endlessly rearranged and remixed, leading to endless tedious (and error-prone) rescripting to regain consistency. So now we just pull in graphical elements as needed. This makes things a bit tricky, since for any particular graph it's not immediately obvious what made that graph, but it's usually not too hard to figure out with some simple searching for filenames (and some verbose naming conventions).<br />
<br />
The other thing is why have the extraction scripts separated from the plots? Well, in practice, the raw data is just too huge to distribute easily this way, and if it was all mushed together with the code and intermediates, it would be hard to distribute. But, at least in our case, the more important fact is that most people don't really care about the raw data. They trust that we've probably done that part right, and what they're most interested are the tables of extracted data. So this way, in the paper folder, we've documented how we pulled out the data along while keeping the focus on what most people will be most interested in.<br />
<br />
[End of nitty gritty here.]<br />
<br />
And then, of course, figure scripting, the topic that brought this whole thing up in the first place. A few thoughts. I get that in principle, scripting is great, because it provides complete documentation, and also because it potentially cuts down on errors. In practice, I think it's hard to efficiently make great figures this way, so we've chosen perhaps a slightly more tedious and error prone but flexible way to make our figures. We use scripts to generate PDFs or EPSs of all relevant graphical elements, typically not spending time to optimize even things like font size and so forth (mostly because all of those have to change so many times in the end anyway). Yes, there is a cost here in terms of redoing things if you end up changing the analysis or plot. Claus Wilke argued that this discourages people from redoing plots, which I think has some truth to it. At the same time, I think that the big problem with figure scripting is that it discourages graphical innovation and encourages people to use lazy defaults that usually suffer from bad design principles—indeed, I would argue it's way too much work currently to make truly good graphics programmatically. Take this example:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-2_1VPwIzatU/WYJzbEkreII/AAAAAAAAJww/_vkDGvqUXAAynnCC4OjiE4V8J_1u10PNQCLcBGAs/s1600/Screen%2BShot%2B2017-08-02%2Bat%2B8.48.50%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="871" data-original-width="1600" height="348" src="https://3.bp.blogspot.com/-2_1VPwIzatU/WYJzbEkreII/AAAAAAAAJww/_vkDGvqUXAAynnCC4OjiE4V8J_1u10PNQCLcBGAs/s640/Screen%2BShot%2B2017-08-02%2Bat%2B8.48.50%2BPM.png" width="640" /></a></div>
<br />
<br />
Or imagine writing a script for this one:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-RdJ_Ku3GSUQ/WYJzlxmuZAI/AAAAAAAAJw0/UTkRPm3xi2chXziSC_NsUMX7VPYaPxOJgCLcBGAs/s1600/Screen%2BShot%2B2017-08-02%2Bat%2B8.49.06%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1258" data-original-width="1600" height="500" src="https://3.bp.blogspot.com/-RdJ_Ku3GSUQ/WYJzlxmuZAI/AAAAAAAAJw0/UTkRPm3xi2chXziSC_NsUMX7VPYaPxOJgCLcBGAs/s640/Screen%2BShot%2B2017-08-02%2Bat%2B8.49.06%2BPM.png" width="640" /></a></div>
<br />
Maybe you like or don't like these type of figures, but either way, not only would it take FOREVER to write up a script for these (at least for me), but by the time you've done it, you would probably never build up the courage to remix these figures the dozen or so times we've reworked this one over the course of publication. It's just faster, easier, and more intuitive to do with a tool for, you know, playing with graphical elements, which I think encourages innovation. Also, many forms of labeling of graphs that reduce cognitive burden (like putting text descriptors directly next to the line or histogram that they label) are much easier in Illustrator and much harder to do programmatically, so again, this works best for us. It does also, however, introduce a human element for error, and that has happened to us, although I should say that programmatic figures are a typo away from errors as well, and that's happened, too. There is also the option to link figures, and we have done that with images in the past, but in the end, relying on Illustrator to find and maintain links as files get copied around just ended up being too much of a headache.<br />
<br />
Note that this is how we organize final figures, but what about exploratory data analysis? In our lab, that ends up being a bit more ad-hoc, although some of the same principles apply. Following the full strictures for everything can get tedious and inhibitory, but one of the main things we try and encourage in the lab is keeping a computational lab notebook. This is like an experimental lab notebook, but, uhh, for computation. Like "I did this, hoped to see this, here's the graph, didn't work." This has been, in practice, a huge win for us, because it's a lot easier to understand human descriptions of a workflow than try and read code, especially after a long time and double especially for newcomers to the lab. Note: I do not think version control and commit messages serve this purpose, because version control is trying to solve a fundamentally different problem than exploratory analysis. Anyway, talked about this computational lab notebook thing <a href="http://rajlaboratory.blogspot.com/2016/03/from-over-reproducibility-to.html" target="_blank">before</a>, should write something more about it sometime.<br />
<br />
One final point: like I said, one of the main benefits to these sorts of workflows is that they help minimize mistakes. That said, mistakes are going to happen. There is no system that is foolproof, and ultimately, the results will only be as trustworthy as the practitioner is careful. More on that in another post as well.<br />
<br />
Anyway, very interested in what other people's workflows look like. Almost certainly many ways to skin the cat, and curious what the tradeoffs are.ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com2tag:blogger.com,1999:blog-5506135718533366764.post-90557956961405868262017-07-30T12:31:00.000-04:002017-07-30T13:07:46.362-04:00Can we measure science?I was writing a couple grants recently, some with page limits and some with word limits. Which of course got me thinking about the differences in how to game these two constraints. If you have a word limit, you definitely don’t want to use up your limit on a bunch of little words, which might lead to a bit more long-wordiness. With the page limit, though, you spend endless time trying to use shorter words to get that one pesky paragraph one little line shorter (and hope the figures don’t jump around). Each of these constraints has its own little set of games we play trying to obey the letter of the law while seemingly breaking its spirit. But here’s the thing: no amount of "gaming the system" will ever allow me to squeeze a 10 page grant into 5 pages. While there’s always some gamesmanship, in the end, <i>it is hard to break the spirit of the metric</i>, at least in a way that really matters. [Side note, whoever that reviewer was who complained that I left 2-3 inches of white space at the end of my last NIH grant, that was dumb—and yes, turns out the whole method does indeed work.]<br />
<br />
I was thinking about this especially in the context of metrics in science, which is predicated on the idea that we can measure science. You know, things like citations and h-index and impact factor and RCR (NIH’s relative citation ratio) and so forth. All of which many (if not most) scientists these days declare as being highly controversial and without any utility or merit—"Just read the damn papers!" is the new (and seemingly only) solution to everything that ails science. Gotta say, this whole thing strikes me as surprisingly <i>un</i>scientific. I mean, we spend our whole lives predicated on the notion that carefully measuring things is the way to understand the world around us, and yet as soon as we turn the lens on ourselves, it’s all “oh, it’s so horribly biased, it’s a popularity contest, all these metrics are gamed, it’s there’s no way to measure someone’s science other than just reading their papers. Oh, and did I mention that time so and so didn’t cite my paper? What a jerk.” Is everyone and every paper a special snowflake? Well, turns out you can <a href="http://iopscience.iop.org/article/10.1088/0034-4885/68/4/R03/meta" target="_blank">measure snowflakes</a>, too (Libbrecht's snowflake work is pretty cool, BTW <a href="http://www.smithsonianmag.com/science-nature/the-art-and-science-of-growing-snowflakes-in-a-lab-180949243/" target="_blank">1</a> <a href="http://www.snowcrystals.com/" target="_blank">2</a>).<br />
<br />
I mean, seriously, I think most of us wish we had the sort of nice quantitative data in biology that we have with bibliometrics. And I think it’s reasonably predictive as well. Overall, <b>better papers end up with more citations</b>, and I would venture to say that the predictive power is better than most of what we find in biology. Careers have certainly been made on worse correlations. But, unlike the rest of biomedical science, any time someone even insinuates that metrics might be useful, out come the anecdotes:<br />
<div>
<ul>
<li>“What about this undercited gem?” [typically one of your own papers]</li>
<li>“What about this overhyped paper that ended up being wrong?” [<i>always</i> someone else’s paper]</li>
<li>“What about this bubble in this field?” [most certainly not your own field]</li>
</ul>
<div>
Ever see the movie “Minority Report”, where there are these trio of psychics that can predict virtually every murder, leading to a virtually murder-free society? And it’s all brought down because of a single case the system gets wrong about Tom Cruise? Well, sign me up for the murder-free society and send Tom Cruise to jail, please. I think most scientists would agree that self-driving cars will lead to statistically far fewer accidents than human-driven cars, and so even if there’s an accident here and there, it’s the right thing to do. Why doesn’t this rational approach translate to how we think about measuring the scientific enterprise?</div>
<div>
<div>
<br />
Some will say these metrics are all biased. Like, some fields are more hot than others, certain types of papers get more citations, and so forth. Since when does this mean we throw our hands up in the air and just say “Oh well, looks like we can’t do anything with these data!”? What if we said, oh, got more reads with this sequencing library than that sequencing library, so oh well, let’s just drop the whole thing? Nope, we try to correct and de-bias the data. I actually think NIH did a pretty good job of this with their relative citation ratio, which generally seems to identify the most important papers in a given area. <a href="https://icite.od.nih.gov/">Give it a try</a>. (Incidentally, for those who maintained that NIH was simplistic and thoughtless in how it was trying to measure science during the infamous "Rule of 21" debate, I think <a href="http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002541">this paper explaining how RCR works</a> belies that notion. Let's give these folks some credit.)</div>
<div>
<br /></div>
<div>
While I think that citations are generally a pretty good indicator, the obvious problem is that for evaluating younger scientists, we can't wait for citations to accrue, which brings us to the dreaded Impact Factor. The litany of perceived problems with impact factor is too long and frankly too boring to reiterate here, but yes, they are all valid points. Nevertheless, the fact remains that there is a good amount of signal along with the noise. Better journals will typically have better papers. I will spend more time reading papers in better journals. Duh. Look, part of the problem is that we're expecting too much out of all these metrics (restriction of range problem). Here's an illustrative example. Two papers published essentially simultaneously, one in <a href="https://www.nature.com/nature/journal/v442/n7104/full/nature04974.html" target="_blank">Nature</a> and one in <a href="https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.96.178102" target="_blank">Physics Review Letters</a>, with essentially the same cool result: DNA overwinds when stretched. As of this writing, the Nature paper has <a href="https://scholar.google.com/citations?view_op=view_citation&hl=en&user=i-F4LuwAAAAJ&citation_for_view=i-F4LuwAAAAJ:9yKSN-GCB0IC" target="_blank">280</a> citations, and the PRL paper has <a href="https://scholar.google.com/citations?view_op=view_citation&hl=en&user=5L5sAlUAAAAJ&citation_for_view=5L5sAlUAAAAJ:u-x6o8ySG0sC" target="_blank">122</a>. Bias! The system is rigged! Death to impact factor! Or, more rationally, two nice papers in quality journals, both with a good number of citations. And I'm guessing that virtually any decent review on the topic is going to point me to both papers. Even in our supposedly quantitative branch of biology, aren't we always saying "Eh, factor of two, pretty much the same, it's biology…"? Point is, I view it as a threshold. Sure, if you ONLY read papers in the holy triumvirate of Cell, Science and Nature, then yeah, you're going to miss out on a lot of awesome science—and I don't know a single scientist who does that. (It would also be pretty stupid to <i>not</i> read anything in those journals, can we all agree to that as well?) And there is certainly a visibility boost that comes with those journals that you might not get otherwise. But if you do good work, it will more often than not publish well and be recognized.</div>
<div>
<br /></div>
<div>
Thing is that we keep hearing these "system is broken" anecdotes about hidden gems while ignoring all the times when things actually work out. Here's a counter-anecdote from my own time in graduate school. Towards the end of my PhD, I finally wrapped up my work on stochastic gene expression in mammalian cells, and we sent it to Science, Nature and PNAS (I think), with editorial rejections from all three (yes, this journal shopping is a demoralizing waste of time). Next stop was PLoS Biology, which was a pretty new journal at the time, and I remember liking the whole open access thing. Submitted, accepted, and then there it sat. I worked at a small institute (Public Health Research Institute), and my advisor Sanjay Tyagi, while definitely one of the most brilliant scientists I know, was not at all known in the single cell field (which, for the record, did actually exist before scRNA-seq). So nobody was criss-crossing the globe giving talks at international conferences on this work, and I was just some lowly graduate student. And yet even early on, it started getting citations, and now 10+ years later, it is my most cited primary research paper—and, I would say, probably my most influential work, even compared to other papers in "fancier" journals. And, let me also say that there were several other similar papers that came out around the same time (<a href="https://scholar.google.com/citations?view_op=view_citation&hl=en&user=MYZVrRYAAAAJ&citation_for_view=MYZVrRYAAAAJ:u5HHmVD_uO8C" target="_blank">Golding et al. Cell 2005</a>, <a href="https://scholar.google.com/scholar?cites=1391493436812127618&as_sdt=5,39&sciodt=0,39&hl=en" target="_blank">Chubb et al. Curr Biol 2006</a>, <a href="https://scholar.google.com/citations?view_op=view_citation&hl=en&user=f0j2UocAAAAJ&citation_for_view=f0j2UocAAAAJ:9yKSN-GCB0IC" target="_blank">Zenklusen and Larson et al. Nat Struct Mol Bio 2008</a>), all of which have fared well over time. Cool results (at least within the field), good journals, good recognition, great! By the way, I can't help but wonder if we had published this paper in the hypothetical preprint-only journal-less utopia that seems all the rage these days, would anyone have even noticed, given our low visibility in the field?</div>
<div>
<br /></div>
<div>
So what should we do with metrics? To be clear, I'm not saying that we should <i>only</i> use metrics in evaluation, and I agree that there are some very real problems with them (in particular, trainees' obsession with the fanciest of journals—chill people!). But I think that the judicious use of metrics in scientific evaluation does have merit. One area I've been thinking about is more nefarious forms of bias, like gender and race, which came up in a recent Twitter discussion with Anne Carpenter. Context was whether women face bias in citation counts. And the answer, perhaps unsurprisingly, is yes—check out this <a href="https://www.nature.com/articles/s41550-017-0141" target="_blank">careful study in astrophysics</a> (also <a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0145931" target="_blank">1</a> <a href="http://www.nature.com/news/bibliometrics-global-gender-disparities-in-science-1.14321" target="_blank">2</a> with similar effects). So again, should we just throw our hands up and say "Metrics are biased, let's toss them!"? I would argue no. The paper concludes that the bias in citation count is about 10% (actually 5% raw, then corrected to 10%). Okay, let's play this out in the context of hiring. Let's say you have two men, one with 10% fewer citations than the other. I'm guessing most search committees aren't going to care much whether one has 500 cites on their big paper instead of 550. But now let's keep it equal and put a woman's name on one of the applications. Turns out there are <a href="http://www.pnas.org/content/109/41/16474.full.pdf" target="_blank">studies on that as well</a>, showing a >20% decrease in hireability, even for a technician position, and my guess is that this would be far worse in the context of faculty hiring. I've know of at least two stories of people combating bias—effectively, I might add—in these higher level academic selection processes by using hard metrics. Even simple stuff like counting the number of women speakers and attendees at a conference can help. Take a look at the Salk gender discrimination lawsuit. Yes, the response from Salk about how the women scientists in question had no recent Cell, Science, or Nature papers or whatever is absurd, but notice that the lawsuits themselves mention various metrics: percentages, salary, space, grants, not to mention "glam" things like being in the National Academies as proxies for reputation. Don't these hard facts make their case far stronger and harder to dismiss? Indeed, isn't the fact that we have metrics to quantify bias critical here? Rather than saying "citations are biased, let's not use them", how about we just boost women's cites by 10% in any comparison involving citations, adjusting as new data comes in?</div>
<div>
<br /></div>
<div>
Another interesting aspect of the metric debate is that people tend to use them when it suits their agenda and dismiss them when they don't. This became particularly apparent in the Rule of 21 debate, which was cast as having two sides: those with lots of grants and seemingly low per dollar productivity per Lauer's graphs, and those with not much money and seemingly high per dollar productivity. At the high end were those complaining that we don't have a good way to measure science, presumably to justify their high grant costs because the metrics fail to recognize just how super-DUPER important their work is. Only to turn around and say that actually, upon reanalysis, <a href="https://medium.com/@shane_52681/the-new-nih-rule-of-21-threatens-to-give-up-on-american-preeminence-in-biomedical-research-based-c40060bd3022" target="_blank">their output numbers actually justify their high grant dollars</a>. So which is it? On the other end, we have the "riff-raff" railing against metrics like citation counts for measuring science, only to embrace them wholeheartedly when they show that those with lower grant funding yielded seemingly more bang for the buck. Again, which is it? (The irony is that the (yes, correlative) data seem to argue most for increasing those with 1.5 grants to 2.5 or so, which probably pleases neither side, really.)</div>
<div>
<br /></div>
<div>
Anyway, metrics are flawed, data are flawed, methodologies are flawed, that's <i>all</i> of science. Nevertheless, we keep at it, and try to let the data guide us to the truth. I see no reason that the study of the scientific enterprise itself should be any different. Oh, and in case I still have your attention, you know, there's this one woefully <a href="http://genesdev.cshlp.org/content/30/5/567.short" target="_blank">undercited gem</a> from our lab that I'd love to <a href="https://www.youtube.com/watch?v=qHZySLDR9dg" target="_blank">tell you about</a>… :)</div>
</div>
</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com8tag:blogger.com,1999:blog-5506135718533366764.post-18235163922459842252017-07-04T16:33:00.000-04:002017-07-04T16:33:02.515-04:00A system for paid reviews?Some discussion on the internet about how slow reviews have gotten and how few reviewers respond, etc. The suggestion floated was paid review, something on the order of $100 per review. I have always found this idea weird, but I have to say that I think review times have gotten bad enough that perhaps we have to do something, and some economists have some research showing that paid reviews speed up review.<br />
<br />
In practice, lots of hurdles. Perhaps the most obvious way to do this would be to have journals pay for reviews. The problem would be that it would make publishing even more expensive. Let's say a paper gets 6-9 reviews before getting accepted. Then in order for the journal to be made whole, they'd either take a hit on their crazy profits (haha!), or they'd pass that along in publication charges.<br />
<br />
How about this instead? When you submit your paper, you (optionally) pay up front for timely reviews. Like, $300 extra for the reviews, on the assumption that you get a decision within 2 weeks (if not, you get a refund). Journal maybe can even keep a small cut of this for payment overhead. Perhaps a smaller fee for re-review. Would I pay $300 for a decision within 2 weeks instead of 2 months? Often times, I think the answer would be yes.<br />
<br />
I think this would have the added benefit of people submitting fewer papers. Perhaps people would think a bit harder before submitting their work and try a bit harder to clean things up before submission. Right now, submitting a paper incurs an overhead on the community to read, understand and provide critical feedback for your paper at essentially no cost to the author, which is perhaps at least part of the reason the system is straining so badly.<br />
<br />
One could imagine doing this on BioRxiv, even. Have a service where authors pay and someone commissions paid reviews, THEN the paper gets shopped to journals, maybe after revisions. Something was out there like this (Axios Review), but I guess it closed recently, so maybe not such a hot idea after all.<br />
<br />
Thoughts?ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com7tag:blogger.com,1999:blog-5506135718533366764.post-60397996879917357442017-06-30T21:10:00.001-04:002017-06-30T21:10:26.114-04:00#overlyhonestauthorcontributions<div>
___ toiled over ridiculous reviewer experiments for over a year for the honor of being 4th author.</div>
<div>
<br />___ did all the work but somehow ended up second author because the first author "had no papers".</div>
<div>
<br />___ told the first author to drop the project several times before being glad they themselves thought of it.</div>
<div>
<br />___ was better to have as an author than as a reviewer.</div>
<div>
<br />___ ceased caring about this paper about 2 years ago.</div>
<div>
<br /></div>
<div>
<div>
Nobody's quite sure why ___ is an author, but it seems weird to take them off now.</div>
</div>
<div>
<br />___ made a real fuss about being second vs. third author, so we made them co-second author, which only serves to signal their own utter pettiness to the community.</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com0tag:blogger.com,1999:blog-5506135718533366764.post-10722591873259173102017-05-05T23:47:00.000-04:002017-05-05T23:52:26.840-04:00Just another Now-that-I'm-a-PI-I-get-nothing-done dayJust had another one of those typically I-got-nothing-done days. I’m sure most PIs know the feeling: the day is somehow over, and you’re exhausted, and you feel like you’ve got absolutely nothing to show for it. Like many, I've had more of these days than I'd care to count, but this one was almost poetically unproductive, because here I am at the end of the day, literally staring at the same damn sentence I’ve been trying to write since the morning.<br />
<br />
Why the case of writer's block? Because I spent today like most other work days: sitting in the lab, getting interrupted a gazillion times, not being able to focus. I mean, I know what I should do to get that sentence written. I could have worked from home, or locked myself in my office, and I know all the productivity rules I violate on a routine basis. But then I thought back on what really happened today…<br />
<br />
Arrived, sat down, opened laptop, started looking at that sentence. Talked with Sydney about strategy for her first grant. Then met with Caroline to go over slides for her committee meeting—we came up with a great scheme for presenting the work, including some nice schematics illustrating the main points. Went over some final figure versions from Eduardo, which were greatly improved from the previous version, and also talked about the screens he’s running (some technical problems, but overall promising). And also, Eduardo and I figured out the logic needed for writing that cursed sentence. Somewhere in there, watched Sara hit submit on the final revisions for her first corresponding author paper! Meanwhile, Ian’s RNATag-seq data is looking great, and the first few principal components are showing exactly what we want. Joked around with Lauren about some mistake in the analysis code for her images, and talked about her latest (excellent) idea to dramatically improve the results. Went to lunch with good friend and colleague John Murray, talked about kids and also about a cool new idea we have brewing in the lab; John had a great idea for a trick to make the data even cooler. Chris dragged me into the scope room because the CO2 valve on the live imaging setup was getting warm to the touch, probably because CO2 had been leaking out all over the place because a hose came undone. No problem, I said, should be fine—and glad nobody passed out in the room. Uschi showed me a technical point in her SNP FISH analysis that suggests we can dramatically reduce our false-positive rate, which is awesome (and I’m so proud of all the coding she’s learned!). I filled our cell dewar with liquid nitrogen for a while, looks like it’s fully operational, so can throw away the return box. Sydney pulled me into the scope room to look at this amazing new real-time machine learning image segmentation software that Chris had installed. Paul’s back in med school, but dropped by and we chatted about his residency applications for a bit. While we were chatting, Lauren dropped off half a coffee milkshake I won in a bet. Then off to group meeting, which started with a spirited discussion about how to make sure people make more buffers when we run out, after which Ally showed off the latest genes she’s been imaging with expansion microscopy, and Sareh gave her first lab meeting presentation (yay!) on gene induction (Sara brought snacks). Then collaborators Raj and Parisha stayed for a bit after group meeting to chat about that new idea I’d talked about with John—they love the idea, but brought up a major technical hurdle that we spent a while trying to figure out (I think we’ll solve it, either with brains or brute force). And then, sat down, stared at that one half-finished sentence again, only to see that it was time to bike home to deal with the kids.<br />
<br />
So yeah, an objective measure of the day would definitely be, hey, I was supposed to write this one sentence, and I couldn’t even get that done. But all in all, now that I think about it, it was a pretty great day! I think PIs often lament their lack of time to think, reminiscing about the Good Old Days when we had time to just focus on our work with no distractions, that we maybe forget about how lucky we are to have such rich lives filled with interesting people doing interesting things.<br />
<br />
That said, that sentence isn’t going to write itself. Hmm. Well, maybe if I wait long enough…ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com2tag:blogger.com,1999:blog-5506135718533366764.post-13176642479380779712017-05-03T05:25:00.002-04:002017-05-03T05:26:05.177-04:00Quick take on NIH point scale: will this shift budget uncertainty to the NIH?Just heard about the new NIH point scale, and was puzzling through some of the implications. First, quick summary: NIH, in an effort to split the pie more evenly, is implementing a system in which each grant you have is assigned a point value, and you are capped at 21 points (3 R01 equivalents). Other grants are worth less. The consequences of this are of course vast, and I'm assuming most of this is going to be covered elsewhere. I'll just say that I do think some labs are just plain overfunded, so this will probably help with that. Also, it's clear from the point breakdown that some things are incentivized and disincentivized, which probably has some pluses and minuses.<br />
<br />
Anyway, I did start wondering about what life would be like for a big lab working with 3 R01s. One of the realities of running such a lab is budget uncertainty. I remember early on when I started at Penn, a (very successful) senior faculty member took me to lunch and was talking about funding and said, "Jeez, my lab is too big, and I've been thinking about how I got here. Thing is you have a grant expiring and you want to replace it, so you have to submit 3 grants hoping that one will come in, but then maybe you get 2 or even all 3, and now you have to spend the money, and your lab gets too big." Clearly, this is bad, and the new system will really help with that. I guess what will happen is that if you get those 3 grants, then you will only take one of them. And, you may have to give back the rest of the grant you already have so that you don't go over 21. Think about this now from the point of view of the NIH: you're going to have money coming back that you didn't expect, and grants not funded that you thought would be funded. The latter is I suppose easy to deal with (just give it to someone else), but I wouldn't be surprised if the former might cause some budgetary problems. Basically, the fluctuations in funding would shift from the PIs to the NIH. Which I think is on balance a good thing. It makes a lot more sense to have NIH manage a large pool of uncertainty in funding than to have individual scientists try and manage crazy step function changes in funding, which will hopefully allow scientists to have more certainty on how much money to expect moving forward. Nice. But maybe I haven't thought through all the angles here.ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com0tag:blogger.com,1999:blog-5506135718533366764.post-86443338441472872442017-04-22T11:09:00.000-04:002017-04-23T16:04:29.534-04:00What will happen when we combine replication studies with positive-result bias?Just read a nice <a href="https://scientistseessquirrel.wordpress.com/2017/04/17/reproducibility-and-robustness/">blog post from Stephen Heard</a> about replicability vs. robustness that I really agree with. Basically, the idea under discussion is how much effort we should devote to exactly repeating experiments (narrow robustness) vs. the more standard way of doing science, which is everyone does their own version to see whether the result holds more generally (broad robustness). In my particular niche of molecular biology, I think most (though definitely not all, you know who you are!) errors are those of judgement rather than technical competence/integrity, and so I think most exact replication efforts are a waste of time, an argument which many other have made as well.<br />
<br />
In the comments, some people arguing for more narrow replication studies made the point that very little (~0%) of our current research budget is devoted to explicitly to replication. Which got me wondering: what might happen if we suddenly funded a lot of replication studies?<br />
<br />
In particular, I worry about positive-result bias. Positive-result bias is basically the natural human desire to find something new: our expectation is X, but instead we found Y. Hooray, look, new science! Press release, please! :)<br />
<br />
Now what happens when when we start a bunch of studies with the explicit mandate to replicate a previous study? Here, the expectation is now what was already found and so positive-result bias would bias towards a refutation. I mean, let’s face it, people want to do something interesting and new that other people care about. The cancer reproducibility project in eLife provides an interesting case study: most of the press around the publication was about how the results were “muddy”, and I definitely saw a great deal more interest in what didn’t replicate than what did.<br />
<br />
Look, I’m not saying that scientists are so hungry for attention that most, or even more than a few, would consciously try to have a replication fail (although I do wonder about that eLife replication paper that applied what seemed to be overly stringent statistical criteria in order to say something did not replicate). All I’m saying is the same hype incentives that we complain about are clearly aligned with failed replication results, and so we should be just as critical and vigilant about them.<br />
<br />
As for apportionment of resources towards replication, I think that setting aside the question as to whether it’s a good use of money from the scientific perspective (I, like others, would argue largely not), there’s also the question of whether it’s a good use of human resources. Having a student or postdoc work on a replication study for years during their training period is not, I think, a good use of their time, and keeps them from the more valuable training experience of actually, you know, doing their own science—let alone robbing them of the thrill of new discovery. Perhaps such studies are best left to industry, which is where I believe they already largely reside.ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com1tag:blogger.com,1999:blog-5506135718533366764.post-85152029202865365072017-04-08T13:32:00.002-04:002017-04-08T21:08:04.089-04:00The hater’s guide to (experimental) reproducibility(Thanks to Caroline Bartman and Lauren Beck for discussions.)<br />
<br />
<b>Okay, before I start, I just want to emphasize that my lab STRONGLY supports computational reproducibility, and we have released data + code (code all the way from raw data to figures) for all papers primarily from our lab for quite some time now. Just sayin’. We do it because a. we can; b. it enforces a higher standard within the lab; c. on balance, it’s the right thing to do.</b><br />
<br />
All right, that said, I have to say that I find, like many others, the entire conversation about reproducibility right now to be way off the rails, mostly because it’s almost entirely dominated by the statistical point of view. My opinion is that this is totally off base, at least in my particular area of quantitative molecular biology; <a href="http://rajlaboratory.blogspot.com/2016/06/reproducibility-reputation-and-playing.html">like I said before</a>, “If you think that github accounts, pre-registered studies and iPython notebooks will magically solve the reproducibility problem, think again.” Yet, it seems that this statistically-dominated perspective is not just a few Twitter people sounding off about Julia and Docker. This "science is falling apart" story has taken hold in the <a href="https://www.wsj.com/articles/the-breakdown-in-biomedical-research-1491576749">broader</a> <a href="https://arstechnica.com/science/2017/04/how-sloppy-science-creates-worthless-cures-and-wastes-billions/">media</a>, and the fact that someone like Ioannidis was even being mentioned for director of NIH (!?) shows how deeply and broadly this narrative has taken hold.<br />
<br />
Anyway, I won’t rehash all the ways I find this annoying, wrongheaded and in some ways dangerous, I’ll just sum up by saying I’m a hater. But like all haters, deep down, my feelings are fueled by jealousy. :) Jealousy because I actually deeply admire the fact that computational types have spent a lot of time thinking about codifying best practices, and have developed a culture and sense of community standards that embodies those practices. And while I do think that a lot of the moralistic grandstanding from computational folks around these issues <a href="http://rajlaboratory.blogspot.com/2016/01/thoughts-on-nejm-editorial-whats-good.html">is often self-serving</a>, that doesn’t mean that talking about and encouraging computational/statistical reproducibility is a bad thing. Indeed, the fact that statisticians dominate the conversation is not their fault, it’s ours: why is there no experimental equivalent to the (statistical/computational) reproducibility movement?<br />
<br />
So first off, the answer is that there is, with lists of validated antibodies and an increased awareness of things like cell line and mycoplasma contamination and so forth. That is all great, but in my experience, these things journals make you check are not typically the reasons for experimental irreproducibility. Fundamentally, these efforts suffer from what I consider a “checklist problem”, which is the idea that reproducibility can be codified into a simple, generic checklist of things. Like, the thought is that if I could just check off all the boxes on mycoplasma and cell identification and animal protocols, then my work would be certified as Reproducible™. This is not to say that we shouldn’t have more checklists (see below), but I just don’t think it’s going to solve the problem.<br />
<br />
Okay, so if simplistic checklists aren’t the full solution, then what is? I think the crux of the issue actually comes back to a conversation we had with the venerable Warren Ewens a while back about how to analyze some data we were puzzling over, and he said something to the effect of “There are all these statistical tests we can think about, but it also has to pass the smell test.” This resonated with me, because I realize that that at least some of us experimentalists DO teach reproducibility, but it’s more of an experiential learning to try and impart an intuitive sense of what discrepancies to ignore and which to lose sleep over. In particular in molecular biology, where our tools are imprecise and the systems are (hopelessly?) complex, this intuition is, in my opinion, <i>the</i> single most skill we can teach our trainees.<br />
<br />
<div>
Thing is, some do a much better job of teaching this intuition than others. I think that where we can learn from the computational/statistical reproducibility movement is to try and at least come up with some general principles and guidelines for enhancing the quality of our science, even if they can’t be easily codified. And within a particular lab, I think there are some general good practices, and maybe it’s time to have a more public discussion about them so that we can all learn from each other. So, with all that in mind, here’s our attempt to start a discussion with some ideas for experimental reproducibility, ranging from day-to-day to big picture:</div>
<div>
<ol>
<li>Keep an online lab notebook that is searchable with links to protocols and is easily shared with other lab members.</li>
<li>Organize protocols in an online doc that allows for easy sharing and commenting. Avoid protocol "fragmentation"; if a variation comes up, spend the time to build that in as a branch point in the protocol. Otherwise, there will be protocol drift, and others may not know about new improvements.</li>
<li>Annotate protocols carefully, explaining, where possible, which elements of the protocol are critical and why (and ideally have some documentation). This helps to avoid protocol cruft, where new steps get introduced and reified without reason. Often, leading a new trainee through a protocol is a good time to annotate, since it exposes all the unwritten parts of the protocol. Note: this is also a good way to explore protocol simplification!</li>
<li>Catalog important lab-generated reagents (probes, plasmids, etc.) with unique identifiers and develop a system for labeling. In the lab, we have a system for labeling and cataloging probes, which helps us figure out post-facto what the difference is between "M20_probe_Cy3" and "M20_probe_Cy3_usethis". What is hard with this is to develop a system for labeling enforcement. Not sure how best to do this. My system is that I won't order any new probes for a person until all their probes are appropriately cataloged.</li>
<li>Carefully track biologic reagents that are known to suffer from lot variability, including dates, lot numbers, etc. Things like matrigel, antibodies, R-spondin.</li>
<li>Set up a system for documenting little experiments that establish a little factoid in the lab. Like "Oh, probe length of 30 works best for expansion microscopy based on XYZ…". These can be invaluable down the line, since they're rarely if ever published—and then turn from lab memory into lab lore.</li>
<li>Journal length limits have led to a culture of very short and non-detailed methods, but there's this thing called the internet that apparently can store and share a lot of information. I think we need to establish a culture of publicly sharing detailed protocols, including annotating all the nuances and so forth. Check out <a href="https://www.addgene.org/crispr/zhang/" target="_blank">this</a> from Feng Zhang about CRISPR (we also have made an extensive single molecule RNA FISH page <a href="https://sites.google.com/site/singlemoleculernafish/" target="_blank">here</a>).</li>
<li>(Lauren) Track experiments in a log, along with all relevant (or even seemingly irrelevant) details. This could be, for instance, a big Google Doc with list of all similar types of experiments, pointing to where the data is kept, and critically, all the little details. These tabulated forms of lab notebooks can really help identify patterns in those little details, but also serve to show other members of the lab what details matter and that they should be attentive to.</li>
<li>Along those lines, record all your failures, along with the type of failure. We've definitely had times when we could have saved a lot of time in the lab if we had kept track of that. SHARE FAILURES with others in the lab, especially the PI.</li>
<li>(Caroline) Establish an objective baseline for an experiment working, and stick to it. Sort of like pre-registering your experiment, in a way. If you take data, what will allow you to say that it worked or didn't work. If it didn't work, is there a rationalization? If so, discuss with someone, including the PI, to make sure you aren't deluding yourself and just ignoring data you don't like. There are often good reasons to drop bits of data, and sometimes we make mistakes in our judgement calls, but at least get a second opinion.</li>
<li>Develop lab-specific checklists. Every lab has it's own set of things it cares about and that people should check, like microscope light intensity or probe HPLC trace or whatever. Usually these are taught and learned through experience, but that strikes me as less efficient than it could be.</li>
<li>Replicates: What constitutes a biological replicate? Is it the same batch of cells grown in two wells? Is it two separate passages of the same cell line? If so, separated by how much time? Or do you want to start each one fresh from a frozen vial? Whatever your system, it's important to come up with some ground rules for what replicates means, and then stick to it. I feel like one aspect of replication is that you don't want the conditions to be necessarily exactly the same, so a little variability is good. After all, that's what separates a biological replicate (which is really about capturing systematic but unknown variability) from a technical replicate (which is statistically variability).</li>
<li>Have someone else take a look at your data without leading them too much with your hypothesis. Do they follow the same logic to reach the same conclusion? Many times, people fall so in love with their crazy hypothesis that they fail to see the simpler (and far more plausible) boring explanation instead. (Former postdoc Gautham Nair was so good at finding the simple boring explanation that we called it the "Gautham transform" in the lab!)</li>
<li>Critically examine parts that don't fit in the story. No story is perfect, especially in molecular biology, which has a serious "everything affects everything" problem. Often times there is no explanation, and there's nothing you can really do about it. Okay, but resist the urge to sweep it under the rug. Sometimes there's new science in there!</li>
<li>Finally, there is no substitute for just thinking long and hard about your work with a critical mindset. Everything else is just, like I said, a checklist, nothing more, nothing less.</li>
</ol>
</div>
<div>
Anyway, some thoughts, and I'm guessing most people already do a lot of this, implicitly or explicitly. We'd love to hear the probably huge list of other ideas people out there have for improving the quality/reproducibility of their science. Point is, let's have a public discussion so that everyone can participate!</div>
ARhttp://www.blogger.com/profile/13811773097412828786noreply@blogger.com15tag:blogger.com,1999:blog-5506135718533366764.post-14967666421847363042017-04-08T10:46:00.002-04:002017-04-08T10:51:53.537-04:00On criticism-by Caroline Bartman<br /><br />Viewed in a certain light, grad school- all of scientific training- is a process of becoming a good critic. You need to learn to evaluate papers and grants either to make them better, to score/review them, or to try to expand your understanding of the field. However, there are many nuances to being a good critic that were never spelled out in my grad school classes, and that I still try to improve on all the time.<div>
<br /></div>
<div>
<b>0. Seeing the bigger picture:</b> What statement is the paper trying to make? How do you feel about THAT STATEMENT after reading it? Every paper has experiments with shortcomings or design flaws. Does the scientific light shine through in spite of that? Or are the authors over-interpreting the data? This is really the key to criticizing scientific work thoughtfully and productively.<br /><br /><b>1. Compassion:</b> Especially important when evaluating the work of others. One person or group can only do so much, due to time, resources, and experimental considerations. When I was an undergrad never having written a paper, I would go to journal clubs and say things like ‘This was a good paper, but what really would have nailed it would be to use these three additional transgenic mouse strains.’ Not realistic! And devalues the effort that’s already represented in the paper. Before you ask for additional experiments, step back: would those really change the interpretation of the paper? Sometimes yes, often no (goes back to point 0). <br />Plus, consciously noting the good aspects of a paper or grant, and only pointing out limited, specific criticisms will make the author happier! So they will be more likely to adopt your suggestions, and in a way actually facilitates the science moving forward.<br /><br /><b>2. Balance:</b> Comes into play when evaluating work that you would be predisposed to like- such as your own work! But also the work of well-known labs (aka fancy science). I often find myself cutting myself slack I wouldn’t give others. (‘That experiment is really just a control, so it’s a waste of time’, etc. ) Reviewers (and also my PIs, thanks Gerd and Arjun) won’t necessarily see your work in such a rosy light! <br />With fancy science, it’s easy to see that e.g. a statement made in a paper isn’t so well supported by the data, but say ‘They’re experts! They founded this field. They probably know what they’re doing.’ Sometimes true, but sometimes not. Would you feel the same way about the paper if it came from an unknown PI? Plus, a fancy lab actually has the best capacity and manpower to carry out the very best experiments with the newest tech! Maybe they should be subject to even harsher scrutiny in their papers.<br /><br /><b>3. Ignorance:</b> I don’t really know if there’s a good name for this quality. Maybe comfort with uncertainty? You are often called upon to evaluate papers or grants that aren’t in your sub-sub-sub field, and that can instill doubts. Yes, you have to recognize your possible lack of expertise. But you can still have valuable opinions! Ideally papers would be read by scientists outside the immediate field, and help inform their thinking. Plus, while technologies differ, scientific reasoning is pretty much constant. So if an experiment or a logical progression doesn’t make sense, you can say something. The worst thing that could happen is someone tells you you’re wrong. <br /><br />Grad school tends to instill the idea that knowledge is the primary quality required to evaluate scientific work. Partially because young trainees do indeed need to amass some body of understanding in order to ‘get’ the field and make comments. But knowledge is really not enough, and sometimes (point 3) not even necessary! <br /><br />Comment if you have more ideas on requirements for a good scientific critic!</div>
Caroline Bartmanhttp://www.blogger.com/profile/04420722703020468795noreply@blogger.com1