Protocol repository Kickstarter

So my friend Lenny Teytelman has a kickstarter campaign going for a project to make an open protocol repository. A great cause to support!

How to maintain "clean experiments"?

During his presentation on software design and "clean code", Gautham made an excellent point: if we spent half as much effort making our code clear as we did on making our papers clear, our code would be pristine. And it's true: I know that I'm fairly relentless about making figures well and trying to write as clearly as possible. It's an exhausting, iterative process, but I feel like it's really important, and I think it's a good idea to apply the same sensibility to our code, which thankfully Gautham has done.

That got me thinking about what other aspects of our work could benefit from a focus on clarity and cleanliness, leading me to a natural candidate: our data. Currently, I am a stickler for "data trails". Every single figure in every paper lives completely in a folder in which lives a single script which, when run, will generate all the elements of the figure from raw data without any manual input required. This way, when you are wondering exactly where that one funny data point came from, you can (relatively) easily find that one threshold from that one cell from that one experiment that led to that data point. Some of this relies on software design, making sure that the code we use the analyze data keeps track of all the relevant metadata.

But what I haven't focused on or really thought about very much is how best to systematize data at the experiment level. That is, how can we ensure that we log our data appropriately and make sure that we can find what we need when we need it. Yes, of course we use lab notebooks and Google Docs, but it's currently a bit of a mish-mash. What is the best way to do this? I don't know, but here are a few of the important features I think such a system must have:

1. It must have low overhead. If you have to take off your gloves in the heat of an experiment to go make an entry in your Google Doc log, there's a high probability it's not going to happen. Paper notebooks are great at this, computers bad, tablets good. I initially bought netbooks for the lab for this purpose, but nobody used them because they were so crappy and took like 38 minutes just to wake from sleep. Maybe an iPad?
2. It must be searchable. This is the singular failure of the paper notebook, which is otherwise so wonderful in so many ways. If you can search quickly through your old experiments to see which experiments used so and so reagent, it really makes you more productive. This is killer, and Google Docs rules for this. Even on a tablet, though, I don't know how you can combine handwriting with searchability, unless you have good optical character recognition.
3. It must be flexible. It has to somehow be able to quickly capture a large number of different types of experiments. Most electronic lab notebooks are so structured and unwieldy, with so many things to click to define your experiment, that they're just not worth the hassle. Again, freeform, like the paper notebook, makes the most sense here.

So far, the closest thing looks like some sort of tablet app that can translate handwriting to searchable text with a cloud organization option would make the most sense. Anybody know of such a thing?

Changing your mind in math

I just posted about changing your mind, and I remarked that math is immune to this because someone is right. True, in a formal sense. But then I (vaguely) remember the following evolution of responses to something new in math:

1. Your result is wrong.
2. Your result is right, but it is trivial.
3. Your result is implicitly known to everyone in the field.
4. Your result has been well known for many decades.
5. Profs. XYZ had that idea many years ago.
6. I had that idea many years ago.
7. You might be on to something.

Along these lines, a reviewer of one of our papers clearly didn’t like it, but we wore them down, and they eventually relented, saying something like “Let history be the judge”. History has actually treated the paper well, but I thought that was pretty funny!

PS: I could have had this series of responses wrong–I don’t remember how it went exactly. Let me know if I got it wrong.

Changing your mind

Scientists argue with each other. A lot. This is supposed to be a good thing, and I think that in its best form, criticism is one of the best ways to ensure the soundness of an approach or the veracity of a discovery. But I think there is an important nuance here: criticism is only useful if someone is willing to change their mind. Ideally both parties, but at least one of them. If you are unwilling to be persuaded by facts, or even entertain plausible alternatives to your point of view, then you are no different from those talking heads on political talk shows. Scientists are not immune to this, as we are humans and as such have our own mental inertia of what we feel is right and wrong (part of the beauty of math, I suppose, is that there is an ultimate arbiter in the dispute). I certainly have had some notions that I’ve been wedding to long past their expiration date.

As such, I’ve been wondering how to do a better job of being open minded. Which got me thinking a bit more about how the process of changing one’s mind actually happens. I find that I rarely change my mind during the actual discussion. Perhaps this stems from the natural desire to defend one’s own position, which I suppose requires what is on some level a fundamentally irrational belief in said position. Usually, I change my mind after the fact, when I have time to think more carefully about all the different points someone made. Is it possible to change this dynamic to make it faster? Or is this just inherent to the process? I feel like my discussions could sometimes be more productive if I could make it easier for me to readily accept new ideas and thoughts counter to my own preconceptions. Hmm. It is as simple as just being more open minded in more situations? I don't think I'm a particularly closed-minded person, but I could be better, I think...

Interesting related side note about review processes: did you know that at the patent office, you can request an interview with your patent examiner to discuss the case face to face (or voice to voice)? Wouldn’t it be nice to have an actual discussion with paper/grant reviewers? I feel like that would make everything much much better.

An alternative to the content exposition mode of lecturing

- Gautham

At lunch we were having a discussion about the role of lectures in education. There is a lot of talk now about "reversing" (or is it "inverting"?) the classroom and about how lectures are not very valuable in their current form.

I think in some ways that is because we are doing the lectures incorrectly. Most lectures feel like an exposition of content. The "course content" in fact. We talk of the lecture "covering" some part of the course's content. These bits of content are the facts that a student is expected to know by the end of it. It also includes the skill to complete certain tasks.

The idea these days seems to be that a lecture is a poor way to communicate knowledge, and that the skill to complete certain tasks is best obtained by the student "doing."

I get fairly heated up when I perceive that "learning by doing" is being put on a higher pedestal than the alternative. Part of my angst is because I am exposed often to situations where a person, sometimes my past self, knows how to "do" something and has been doing it for years, but does it all wrong: Science that is gravely flawed because folks valued doing way over anything else, ugly computer programs, poor lindy hop/balboa on the dance floor, and dangerous and inefficient weightlifting technique in the gym. Everyone is nominally "completing task" but many do it very poorly. I also perceive very clearly how my skill in those four things have been dramatically improved by instruction of various forms.

The issue is that the counterpart to "learning by doing" is usually "learning by listening to content exposition," and that alone has not helped me all that much since I am good at absorbing content alone at a much faster rate from books.  Instead, I have gained tremendously from "observing a master at work and listening to their advice."

So if you are a lecturer, rather than teaching content, you can teach how to do your craft excellently. You can show how you work through a problem as a master of your craft. You can give advice on the wrong roads to take, as those options arise, which you know from your years of experience and constant self-improvement don't take you to a good place. You can show exactly how to hold the hammer so that the nail goes in straight.

If your craft is the development of ideas, you will devote your lecture to that, and there you have reason to go through lengthy derivations, but you will do so to teach your students how to derive new certainties efficiently and how to play with concepts.  I will gain from seeing your approach, and without thinking I will emulate the parts that speak to me, like I subconsciously try to emulate the posture of my dance floor and lifting platform heroes.

  In some ways this makes lecturing easier and in other ways harder. Teaching content is often somewhat boring for the lecturer too, but teaching the lessons one has learned while on the road to mastery comes naturally and with pride. On the other hand one has to be a master to teach like this.

Shortcomings of the GUIDE-inspired approach to programming Matlab GUIs

- Gautham

It is very difficult to make a Matlab GUI that can be reused and comprehended easily (i.e. clean) using the standard approach to building them. This approach is I believe inspired by the type of code produced by GUIDE, Matlab's automatic GUI code generation tool. GUIDE starts from a GUI layout made by the user and creates a code template for it. This approach necessarily puts the GUI front and center, to the detriment of the application design. In most important applications, the GUI should be a detail, not the organizing principle.

To illustrate, here is a code example for one of the simplest GUIs one could imagine: a click counter.

A GUI that counts the number of times the button has been clicked

Imagine that instead of counting clicks, the application is showing you a view of your carefully collected experimental data, and as you press buttons and click here and there, it makes modifications to your data and overwrites them in the disk. In other words, imagine that you care about what the application does.

Before going into the code for how to make such a GUI, first a few helper functions to lay out the GUI components.
Contents of gpndemos.makeFigure.m:

function figH = makeFigure()
    figH = figure('Position', [1 1 200 200]);
end

Content of gpndemos.makeClickMeButton.m

function uihandle = makeClickMeButton()
    uihandle = uicontrol('Style','pushbutton', ...
        'Position', [50 125 100 50], ...
        'String', 'Click me!');
end

Contents of gpndemos.makeTextBox.m

function uihandle = makeTextBox()
    uihandle = uicontrol('Style', 'text', ...
        'Position', [50 50 100 50], ...
        'FontSize', 40);
end

The code to launch the GUI is in gpndemos.makeClassicMatlabGUI.m (note that the second function, buttonCallback, is a local function contained within the file that defines makeClassicMatlabGUI):

function makeClassicMatlabGUI()
    figH = gpndemos.makeFigure();
    Hs.textbox = gpndemos.makeTextBox();
    button = gpndemos.makeClickMeButton();

    Hs.numberOfTimesClicked = 0;
    guidata(figH, Hs)

    set(button, 'Callback', @buttonCallback);
end

function buttonCallback(hObject, eventdata)
    Hs = guidata(hObject);
    Hs.numberOfTimesClicked = Hs.numberOfTimesClicked + 1;
    set(Hs.textbox, 'String', ...
        num2str(Hs.numberOfTimesClicked))
    guidata(hObject, Hs);
end

Then you run from the command line:

>> gpndemos.makeClassicMatlabGUI()

and the GUI will pop up and you can click the button and it will update the display.

It was a mystery to me initially how the single file above can produce a working GUI that persists after the function executes. The main function does three things. First it builds the GUI elements. Then it stores some data in the figure itself, and lastly it sets the action that should occur when the user presses the "Click me!" button. The second and third commands are the mysterious ones. A Matlab figure can be associated with a piece of data called the "guidata" - this thing is a regular Matlab structure (struct) and in our lab the convention is to give it the variable name "Hs". The lines:

Hs.textbox = gpndemos.makeTextBox();
Hs.numberOfTimesClicked = 0;
guidata(figH, Hs)

make sure that someone who has access to the figure will be able to figure out how many times it has been clicked on and the address of the textbox. That interested party is the function that executes when you press the button. Speaking of which, here is the line that tells Matlab what to do when the button is pressed:

set(button, 'Callback', @buttonCallback);

What is intriguing about this is that the function "buttonCallback" is a local function within the "makeClassicMatlabGUI" file, and you can never call it yourself from the command line or any other program you write, unlike the main makeClassicMatlabGUI() function. However, Matlab can call it when you click on the button. Matlab's rule is approximately that a GUI Callback can be set to any function that is "in scope" (accessible) at the moment you do the set(…) operation itself. In our program, we set the button's 'Callback' while running the "makeClassicMatlabGUI" file, and the buttonCallback local function is certainly available at that time. By this mechanism, the button can execute a local function within makeClassicMatlabGUI long after makeClassicMatlabGUI() has finished executing.

Looking above at the code for buttonCallback, it does the following: First, it "beams down" the struct of data held by the figure. Then it increments the click count by one, and then it tells the textbox to display the new click count. Lastly, it "beams up" the updated struct to the figure.
There are many ways to explain the shortcomings of this scheme when building larger applications, but they are all consequences of the fact that the application is trapped within the GUI:

The GUIDE-inspired GUI design traps the application within the GUI

The "business logic" of the program is inaccessible from anywhere outside the GUI. In our example the business logic is just storing and incrementing a counter. There is nothing "GUI" about the concept of storing and incrementing a counter, but the standard design traps that idea within the GUI. To add features to this application you must directly modify the code, since there is no easy way to use or manipulate this application from another program. And since there is no easy way to manipulate the application from another program, there is no easy way to write tests for the business logic.
The first step to a better design is making the application directly accessible to the user, or at least to the programmer:

A freed application is accessible to the user directly.

GUIs are not the only interactive programs one can build in Matlab. In fact, Matlab has an ample and easy to use set of tools to create clean, reusable interactive program through its support for object oriented programming. We'll see how to do that for our "Click me!" example in a next blog post.

Where to find the truth

Gautham gave a awesome group meeting today all about software design. In the course of his presentation, we finally found where the truth is.

(Quote taken from Robert Martin's "Clean Code", p. 54.)

How do you know you're an adult?

I was just shoveling the snow the other day, which always seems to lead to some sort of internal reflection. I was thinking about being an adult. I think I qualify now, mostly, and I was wondering what some of the criteria are. Here’s a few thoughts I had:

You’re an adult if...

1. You’re no longer automatically better at everything just because you’re two years older.

2. You no longer want to be like everyone else, but rather wish you weren’t just like everyone else.

3. You no longer miss your mom when you’re sick.

4. Growing out of your clothes is no longer a good sign. (Sydney)

5. You would love to debate existentialism in early 20th century film, but these dishes sure as hell aren't washing themselves.

Any other thoughts?

What not to worry about when you submit your paper

When I was a graduate student, I used to really worry about the little details of manuscripts when submitting a new paper, like whether it fits in the length requirements or specific figure lettering requirements or how the figures might fit on a page or whatever. I think I know why: it felt like something I could control in an otherwise very opaque and essentially random process. I now think it really just doesn't matter. Yes, try and keep to the basic spirit of the journal (a 50 page treatise on the mathematical details of burst modeling probably doesn't belong at Science), and get some very basic things right like citation formatting, but beyond that, don't sweat the length limits and so forth. Your job at this stage is to convince the editors that it's interesting and to convince the reviewers that its interesting and sound. Reviewers rarely read the supplement, so I when in doubt, put it in the main figure. If you're lucky enough to get your paper accepted or close to it, then great! Now you can worry about all those details, hopefully with some help from the editors.

So write clearly and compellingly, make attractive and easy to parse figures, and don't worry about the small stuff. Hopefully, you'll have plenty of time for that later!

Was it luck?

I think I’ve written about this topic before, but I just saw this TED talk that got me thinking about it again. In this talk, the speaker describes an experiment in which he sets up a rigged Monopoly game between two randomly selected contestants, rigged in the sense that one of the contestants gets way more money and so forth than the other. The interesting thing is that even though both contestants plainly know that the game is rigged and that’s why the rigged contestant wins, when questioned about why she or he won, the winner will say that it was due to their good strategy or good moves or whatever. Apparently, this sort of rationalization is a common psychological reaction.

I wonder if the same thing is at play in scientists. I think that if you ask most successful scientists, they would say that they succeeded due to hard work and a bit of talent, perhaps even pointing out a couple of particular insights that they made along the way. Some might say luck, but probably many of them don’t really mean it. But when look at my own career so far, I have to admit that whatever modicum of success I’ve had is perhaps more stumbled into than earned. I mean, how many choices did I really make, at least that were really consequential? How many were instead the product of the sheer luck of someone knowing someone in the right place at the right time? They say fortune favors the prepared mind, but I think it’s even more true that fortune favors the fortunate.

Apple, Google, and privacy

There seems to be a notion out there that Apple is better than Google when it comes to privacy. Maybe, can't say that I follow that stuff too carefully. But I'm wondering how much of that is because of Apple's utter inability to deliver competent online services than anything else...

Some thoughts on time management

Life is busy. And as I've gotten older, it's gotten busier, to the point where I need a strategy to manage all the stuff I'm supposed to do. Here's are some general thoughts and some specifics of what I've found that works for me in case this is useful to someone (most of this comes from workshop for young faculty I did some time ago that Uri Alon had organized–thanks Uri!).

The main problem that we all face is a large number of meetings and administrative tasks that clutter up our mind, preventing us from doing more useful and meaningful things. A few observations:

1. Once you have a lot of tasks, there is a lot of cognitive overhead associated with remembering the tasks. Like "oh yeah, I'm supposed to e-mail that person about the seminar." Multiply that by 100 (or 1000) and you get the average faculty member's academic life these days. To do lists are valuable because you can spend less time stressing about what you should remember to do and more time just doing them. But they can rapidly become overwhelming and unmanageable if not used with care.
2. Most of us are very bad at prioritizing these tasks.  Eisenhower had it right when he said "What is important is seldom urgent and what is urgent is seldom important."
3. Technology has made it so that one can deal with a lot of administrative stuff without administrative support. However, while technology has facilitated this ability, others have taken advantage of it to inundate us with huge numbers of requests to do things like scheduling meetings, etc. These would have seemed unreasonable in the past, but now that the overhead to ask someone to do something is very low, we have a problem.
4. The feeling of a lack of control over how we spend our time largely stems from our desire to please others, leading us to agree to do too many things. There's a good reason for this, because being a responsive member of a social group, however defined, is a good thing. The key is how to have your cake and eat it too.

In order to deal with this, I have come up with the following system, which I guess is more a set of guidelines than anything else.

First up is the to do list. I have used a to do list since I was a postdoc. But it became completely unmanageable as things progressed. I use the "A, B, C, D task" system, where you have a 2x2 grid of important/not important, urgent/not urgent and put tasks in them. By FAR the biggest problem for me is distinguishing the B vs. C task, which is [important, not urgent] vs. [not important, urgent].

1. "A tasks" (important, urgent): grant due by deadline, prepare lecture for class at noon. These are things that would be catastrophic if you didn't do them.
2. "B tasks" (important, not urgent): study literature for a new research direction, read that highly relevant paper, come up an experimental strategy, finish writing manuscript. These are things that lay the groundwork for the future.
3. "C tasks" (not important, urgent): file reimbursement paperwork, review a paper for a journal, send that collaborator some images from that project. You should do it, but the world won't end if you don't.
4. "D tasks" (not important, not urgent): Write a blog post. :)

I use a piece of software for this called OmniFocus. It has a bewildering array of options, but for me, I organize tasks by project and then have a context for A, B, C and D. That is working well for me, especially since it has a system-wide hotkey for adding in tasks.

A few comments: if every task is an A task, you're doing it wrong. That's a recipe for being overwhelmed. Taking control of your time means making choices. It's hard, and I'm still working on doing a better job of it. But it's essential.

The B/C distinction is a big challenge, both in assignment and execution. B tasks are, by definition, more important, but seldom give the quick little adrenaline rush of completing a task that you can tick off. C tasks are those little things staring you in the face all day, like responding to some e-mail or filling out a Doodle poll for a thesis committee time, that give you the sense that you're accomplishing stuff in the moment, but leave you with that empty feeling at the end of the day (you know what I'm talking about). The temptation is to finish off all the C tasks to have a "clean plate" for doing A and B tasks. Avoid this urge. C tasks are sponges for time: they take up whatever time you give them. So limit the time you spend on them–remember, they are NOT IMPORTANT. Also, many of these requests, while well-meaning, simply would not have existed a decade ago, and the world still went around. A surprising number of these will just resolve themselves, like a "delegation mirror". Remember also that many of your C tasks may have been someone else's A task. The person who needs feedback on the seminar schedule will write to you at 10am about it because it's important to them. Which is fine, but this doesn't mean that it has to be important to you. You can do it later.

This leads to the next point about strategies for making the most out of your day. I function best in the morning. So I try to reserve morning for things that are my A and B tasks. For me, this means interacting with my group and some important writing tasks like manuscripts and grants. I try and avoid the temptation to do those silly e-mails and forms and whatever. After lunch, I'm a bit... less... sentient. So I try and schedule other stuff then. For me, the strategy is clear: PROTECT MY MORNINGS at all costs, and try to make the most of them. One way to do this is to explicitly block off time in your calendar for A/B tasks in the morning (or whatever your "go time" is). It's important to you, and so you have to commit to it. Think of it this way: if someone out of the blue wrote to you asking you for a protocol in the middle of a meeting with a collaborator, would you respond to it right then? Then why do it when you're in the middle of quietly thinking about a project? The point is to reclaim control of your day for you to do the things you want to do, which is presumably what you were hired for anyway.

The other main bugaboo is the dreaded endless e-mail reel. I keep my e-mail on all day long in case something important comes through, but I try and deal with the vast majority of the others on the morning train ride (~15 minutes) or in the evening. During the day, I just try to stick to the ones that really really matter (to me). It seems to be working okay, actually.

Anyway, those are some thoughts on some time management, and there are tons of other approaches to this as well. Probably most senior faculty have already intuited and incorporated many of these suggestions into their day to day life, but I offer them to anyone who's feeling a bit overwhelmed by the number and variety of demands on their time and is looking for some way to deal with it.

Oh, and one other thing. A lot of this thinking goes in reverse. So before sending that e-mail asking for 20 people to figure out dates for a conference call, just be mindful that that single e-mail is generating work for a whole lot of people, most of whom are probably just as busy as you are.

Gettin' scooped

Was just thinking the other day that the only thing worse than opening the Nature Table of Contents and finding out you got scooped is... opening up the Biophysical Journal Table of Contents and finding out you got scooped.

Not that this has happened to us. Yet.