My favorite new tool: ack

If you're like me, about once a week you type something like

>> grep words

and then wait a few seconds before remembering that you really meant

>> grep words *

or perhaps

>> grep -r words *

If so, do yourself a favor and install ack right now. You'll be glad you did.

Note: ack tries to intelligently search only filetypes that it believes to be text. Unfortunately, its built-in list of extensions for such files is incomplete. For instance, it does not include .rst (ReStructured Text) files. To add more extensions, just create a file ".ackrc" in your home directory and put the following line in it:

--type-add=TYPE=.ext

Here TYPE is the name of the filetype (anything you want) and .ext is the extension of that filetype. For example:

--type-add=RST=.rst

(in the line above, "type" is preceded by two minus characters. Unfortunately, they look strange in my blog's font).

Why I won't be a Mendeley university advisor (for now)

Mendeley, my tool of choice for bibliographic reference management and sharing, just announced their "University Advisor" program. Basically, they're asking academics to advocate for them on campus in exchange for free premium accounts.

I already advocate for Mendeley unofficially, because I think it is useful and its usefulness will grow in proportion to the number of people who adopt it. But I'd rather not be officially associated with Mendeley, mostly because it's not yet a sufficiently polished product.

Mendeley is a useful product with a lot of potential, but it still has a lot of serious problems. Some of these seem like telltale signs that the basic software and data infrastructure on which Mendeley is built has fundamental and dangerous flaws.

For instance, my Mendeley collection has 516 documents according to both the Desktop app and the web app. But the "my library stats" page says I have only 328 articles. This doesn't cause any real problems, but the idea that somehow a count of these items is being maintained in a way that it can get permanently out of sync is disturbing. Furthermore, I had an e-mail exchange with Mendeley support about this a couple of months ago, and they admitted it was a problem but they've been unable to solve it.

Duplicates. This is a huge problem with Mendeley. If I import a document twice, even from the same source, I get duplicates. If I drag a document from my library to a "group" twice, I get duplicates. The latter behavior is really inexcusable, since this operation occurs entirely within Mendeley. It also makes it very painful to use groups (so painful that I've stopped using them). I have a group called "Runge-Kutta stability regions", and I'd like to keep all papers in my library with the tag of the same name in that group. This would be easy if I could just periodically select the tag and drag all papers to the group, but that's a recipe for disaster since I'll end up with duplicates. CiteULike sync is also a disaster, as it generates many duplicates.

Bibtex. Mendeley just doesn't seem to pay enough attention to bibtex-centric user needs. In the desktop app, citation keys are not shown by default. When one right-clicks on a document and selects "copy citation" (with citation style set to bibtex), the citation generated is different than what one gets from "export citation" (the cite keys don't agree). Mendeley mangles bibtex fields on import. When articles are removed from Mendeley, they may persist in the auto-synced bibtex file. Combined with the duplicates problem, this can be a nightmare.

Sharing. Mendeley is supposed to facilitate sharing, but unfortunately it also restricts sharing in one very important way: my library is not publicly accessible, and I cannot make it so. Why not? Do people have something to hide in their library of scholarly references?

I could go on, but you get the idea. Mendeley has a lot of things going for it, not the least of which is a very active development team, and I hope it succeeds in achieving what its developers intend. It's my tool of choice, but as its name indicates, it's still beta.

What is science?

Today I received an e-mail from a collaborator of mine stating

...our community doesn't reward engineering effort but instead scientific advances.

The context was a discussion of what is publishable in scientific software development. This got me thinking about what really is the difference between science and engineering, and what is the difference between publishable advances and the rest of the work that scientists do.

After some thought I've concluded that this difference is in many respects artificial and purely a function of one's discipline (or even sub-discipline). To take algorithmic complexity as an example:

-To a complexity theorist, a reduction in complexity from 1000N^3 to 3N^2 would be considered "engineering effort": after all, either one is in P, right?

-To an applied mathematician, the above result would be considered a fabulous scientific advance, but a further reduction from 3N^2 to 2N^2 might not be deemed publishable.

-To a chemist, say, who needs to run the algorithm with 10000 different parameter sets, even a 10% improvement might be considered a valuable scientific advance.

Of course, to a pure mathematician any algorithm for solving the problem is an engineering detail; the only scientific aspect is proving the existence of a unique solution.

All of these advances may be important steps leading from an abstract idea to a technological breakthrough that benefits non-scientists. I think it's okay that all the above researchers are only interested in one particular step in this chain of advances. What's detrimental, however, is the inability to recognize that the other steps in this chain are valuable. This kind of bigotry is, in my experience, rather common among scientists.

Edit: Ironically, the quote at the beginning came from a person whose title is "Professor of Engineering".