Just listened to this Planet Money podcast all about hating on the Dow Jones Industrial Average. Gist of it: the Dow Jones calculates its index in a weird (and most certainly nonsensical) way, and is an anachronism that must die. They also say that no market "professional" (quote added by me) ever talks about the dow, but measures like the S&P 500 and the Wilshire 5000 are far more sensible.
This strikes me as a criticism that distracts from the real issue, which is whether one should be using any stock market indicator as an indicator of anything. Sure, the Dow is "wrong" and the S&P 500 is more "right" in that they weight by market cap. Whatever. Take a look at this:
Pretty sure that this fact goes back longer as well, but Wolfram Alpha only goes back 5 years and I've already wasted too much time on this. Clearly, also, short term fluctuations are VERY strongly correlated—here's the correlation with the S&P 500 in terms of fluctuations:
So I think the onus is on the critics to show that whatever differences there are between the S&P and the Dow are meaningful as predicting something about the economy. Good luck with that.
Of course, as an academic, far be it from me to decry the importance of doing something the right way, even if it has no practical benefit :). That said, in the podcast, they make fun of how the Dow talks about its long historical dataset as an asset, one that outweighs its somewhat silly mode of computation. This strikes me as a bit unfair. Given the very strong correlation between the Dow and S&P 500, this long track record is a HUGE asset, allowing one to make historical inferences way back in time (again, to the extent that any of this stuff has meaning anyway).
I think there are some lessons here for science. I think that it is of course important to calculate the right metric, e.g. TPM vs. FPKM. But let's not lose sight of the fact that ultimately, we want these metrics to reflect meaning. If the correspondence between a new "right" metric and an older, flawed one is very strong, then there's no a priori reason to disqualify results calculated with older metrics, especially if those differences don't change any *scientific* conclusions. Perhaps that's obvious, but I feel like I see this sort of thing a lot.