7 Habits of the Open Scientist #3: Pre-publication dissemination of research

Note: this post is part of a series on habits of the open scientist.  Here I discuss the third habit, pre-publication dissemination of research.  The previous post was on reproducible research.

A personal story

In 2003-2004, as a senior undergraduate, I got involved in research on strong stability preserving (SSP) Runge--Kutta (RK) methods.  I noticed a number of "numerical coincidences" -- certain numbers characterizing ostensibly different properties of RK methods always happened to be exactly the same.  I didn't yet have the necessary background to fully prove the conjectured connection, but after months of work, I finally succeeded in completing a partial solution to the problem, which I wrote up as my undergraduate thesis.  Before I could submit a manuscript for publication, I discovered that two other researchers had just published the full result.  Hence my manuscript was, of course, unpublishable.

Occasionally, situations like this are inevitable.  But those researchers had worked out and written up the result at least a year ahead of me -- before I even began the work in earnest.  If their work had been available to me at the outset, I could have devoted my time to unanswered questions.

Refereeing is slow; distribution is fast

In my field (applied math), it often takes more than 1 year for a submitted paper to be published.  This is because a thorough referee process of a manuscript takes time, and I think that time is worthwhile.  In contrast, I can "publish" a new paper on the arXiv in just 48 hours, or on my professional website instantaneously.

Many readers may not wish to see my work until it has been refereed.  But for those working on similar problems, reading my work 1 year earlier can be very useful by pointing out promising new avenues or avoiding duplication of effort.

The open scientist distributes his publishable research openly before the formal refereeing and publishing process, by placing completed manuscripts on a preprint server like the arXiv.

If you're brave, you can also share your grant proposals openly.

Concerns

The first time you do this, you may feel worried that someone else will 'steal' your preprint and publish it before you.  But posting it on the arXiv makes it public and stamps it with a date, so such theft would be obvious to everyone.  You may be worried that others will steal your ideas and immediately begin working on your next planned research question.  But if you're like me, the number of related research questions to pursue is essentially endless and you'd be fortunate if your efforts attract others to work on closely related topics (for the truly self-interested, note that it will increase your citation count).  Finally, in some fields or subfields, there is cultural resistance to making preprints public; you can see my take on the issue in this prior blog post.  But there are signs that it is gaining wider acceptance.

If everyone began to practice this, it would effectively transform the role of journals.  They would no longer be the primary distribution apparatus; their role would be that of filtering the already-published literature.  This would make a lot more science available a lot sooner, without sacrificing the usefulness of peer review. 

Next up: Habit #4 -- Open notebook science.

7 Habits of the Open Scientist

Science has always been based on a fundamental culture of openness.  The scientific community rewards individuals for sharing their discoveries through perpetual attribution, and the community benefits by through the ability to build on discoveries made by individuals.  Furthermore, scientific discoveries are not generally accepted until they have been verified or reproduced independently, which requires open communication.

 

Historically, openness simply meant publishing one's methods and results in the scientific literature.  This enabled scientists all over the world to learn about essential advances made by their colleagues, modulo a few barriers.  One needed to have access to expensive library collections, to spend substantial time and effort searching the literature, and to wait while research conducted by other groups was refereed, published, and distributed.

 

Nowadays it is possible to practice a fundamentally more open kind of research -- one in which we have immediate, free, indexed, universal access to scientific discoveries.  The new vision of open science is painted in lucid tones in Michael Nielsen's Reinventing Discovery.  After reading Nielsen's book, I was hungry to begin practicing open science, but not exactly sure where to start.  Here are seven ways I'm aware of.  Each will be the subject of a longer forthcoming post.

 

I believe that every scientist has a moral imperative to adopt the first two:

 

1. Freely accessible publications.  At a minimum, make sure that everyone is allowed to read your research.

2. Reproducible research.  Release your code and data so  that anyone who wants to can verify or build directly on your work.

 

The remaining five are marks of a truly open scientist:

 

3. Pre-publication dissemination of research.  Just because peer-review and journals take time, that doesn't mean you need to embargo your audience.

4. Open collaboration through social media.  Find the person who knows that one thing you need, through new scientific networking tools -- and share your own expertise where it's needed most.

5. Live open science.  Tell people about your marvelous discoveries -- as you make them.

6. Open expository writing.  Teach others about the field you work in through a blog or online book.

7.  Open bibliographies and reviews.  Let your colleagues know what you're reading, and what you've learned from it.

What is this thing we call the arXiv, really?

When should you submit your papers to the arXiv?  Before or after they are published in a journal?  It turns out that this is a controversial question, with different right answers from different communities.  Consider the following comment from a fascinating discussion on astrobetter.com:

...you want people to be using and thinking about the “final” version of the paper, not just something they found lying around on the internet. It would feel like cheating to me to post to astro-ph before having the paper formally accepted by the journal. I do pick out several people from the reference list whose work I cite...should they read it on astro-ph instead? I’d feel like a jerk...I will never referee a paper that has already been thrown up on astro-ph. If they have that little respect for the refereeing process, fine, let them “publish” on the web. But it doesn’t (and shouldn’t) count. IMNSHO.

(emphasis added)

Contrast this with the following question and answer on MathOverflow:

Q: How do people usually use arxiv to put their papers? At which stage does one usually put his/her paper/report there?

A:...it is standard to post on the arxiv as soon as one is ready to share one's work with colleagues...it is quite common to post on the arxiv at the same time as submitting, or not long prior to submitting. (But there is nothing wrong with posting on the arxiv and then spending some more time polishing your preprint before submitting it to a journal.)

There are many voices on both sides of the question in both of the discussions linked above.  How can we have such different views of how to use the arXiv?  People seem to have two very different concepts of what the arXiv is.  The arXiv itself doesn't tell us what exactly it is meant to be*,  so we're left to create a definition personally or as a community.  The first attitude above seems to be based on the following definition:

A. The arXiv is an official outlet for papers whose quality and correctness has been certified already.  Its purpose is to make those papers available without the need to pay a journal subscription fee.

The second opinion seems to be based on a quite different definition:

B. The arXiv is a one-stop outlet for the latest scientific research in my field.  Its purpose is to disseminate new information as quickly and broadly as possible so that it can then be understood, evaluated, and added to.

Personally, I take viewpoint B, which I think is more common among mathematicians.  What is the arXiv for you?

 

*: the most relevant statement from the arXiv primer is this:

arXiv supplements the traditional publication system by providing immediate dissemination and open access to scholarly articles (which often later appear in conventional journals). It is important to note, however, that arXiv is not a repository for otherwise unpublishable material, nor is it a refereed publication venue.

Endorse the Science Code Manifesto

Read it.
Sign it.
Do it.
http://sciencecodemanifesto.org.

Do you know what your colleagues are reading?

Up until Google's recent (catastrophic) changes to Reader, I used it to share and discuss interesting journal articles.  It was a near-perfect platform for this, and I'm hopeful that we'll have a replacement soon.

The great utility of it was that my colleagues are very good at discerning which articles may be of interest to others in our circle.  This is no surprise, since we have similar research interests.  The fraction of articles that are actually interesting to me, for most journal RSS feeds that I check, is 1%-5%, which means I spend a lot of time scanning article titles.  In contrast, the fraction of papers shared by my colleagues that I find interesting is probably closer to 50%!

I've found a nice way to display a public RSS feed of papers that I read*, via Mendeley (it's shown here on the right).  Now, ideally, Mendeley would allow me to publish a feed that includes all papers in my Mendeley library as I add them.  They don't, but they do something almost as good: they provide a public RSS feed showing all papers for any public Mendeley group, as they are added.  So here's what I did:

1. Create a public Mendeley group for my own library.

2. Whenever I import a new reference to Mendeley, I also add it to the group (note that you can do this via the dropdown menu in the popup that appears whenever you use the 'Import to Mendeley' bookmarklet.

3. I got the address for the feed from Mendeley (log in, click the 'Groups' tab, click 'Papers' on the left, and look for the RSS feed icon on the top right) and added a widget here on my blog, as well as on my professional webpage.

That's it.  If you want to subscribe to this RSS feed, here it is:

feed://www.mendeley.com/groups/1194891/david-ketcheson-s-library/feed/rss/

 

[*] Note that 'read' here means 'read at least the abstract.'

Managing publication lists in HTML

As an academic, it's a good idea to maintain a professional website with a list of your publications.  Ideally, this list should include links to where visitors can download the papers (PDFs) and any related code.  In my case, I also maintain a website for my research group that has another publication list. Of course, you need to maintain local reference files with the citation info for your publications (for inclusion in later publications), as well as your CV.

Maintaining all these separate lists can become very tedious, which is probably why most academics' sites are usually out-of-date.  Here's how I automate much of it:

• I use Bibtex. This allows automatic generation of bibliographies for papers and for my CV.  And it's the basis for generating HTML bibliographies, as outlined below.
• I use Mendeley. Mendeley allows me to create separate bibliographic lists (e.g., for my own publications and for those of my research group) and export them as bibtex files, using the desktop app.  It's also possible to get a bibtex file of your Mendeley publications directly on-line through BibBase.  You can even display a list of your publications on-line automatically using Mendeley, but it's not very customizable; for instance, they appear in alphabetical, rather than chronological order.
• I would use BibBase to automatically display an updated bibliography if my institutional server ran PHP or CGI scripts, but no luck there.
• My less-automatic solution is simple Python scripts that generate CSS-friendly HTML.  The scripts use the Python module bibliograph to parse bibtex entries.  I could instead use bibtex2html.  I like my solution better because the customization is all done using CSS rather than Python, so I can change the look without re-generating the HTML.  Of course, bibtex2html is a more polished and flexible tool than my 30-line script.
• Here's a CSS snippet that's used for this publication list:

#pub {padding: 5px; border-width: 2px; border-style: none; background-color: #eee4b5; font-size: 1.2em; margin-top: 20px; margin-bottom: 20px;}
#pub a{font-weight: bold; color: #09434e;}
#pub name{font-weight: bold; color: #09434e;}
#pub journal{font-style: italic;}

 

The workflow for adding a new paper to an html bibliography is:

1. Add the paper in Mendeley.
2. Export bibtex from Mendeley.
3. Run Python scripts.
4. Paste resulting HTML into the appropriate file.

Again, it would be simpler if I could use Bibbase (cutting out steps 2-4).  It's still fairly painless, and it's easy to generate new bibilographic lists or customize the look of existing ones.

What journals do you read?

As a scientist, one is defined by the kind of problems one works on, the conferences one attends, the journals one publishes in, and the journals one reads. All of these except the last are more or less publicly available information.

Only you know precisely which journals you choose to read, yet they're an essential part of your scientific identity. They determine the kind of new advances you're likely to be aware of and where your research may turn in the future.

I've made my Mendeley library public, so anyone can see in great detail not only what journals I read but which articles I read. But most of you are probably not interested in quite that level of detail, so here's a list of the journals I follow closely. I collect their RSS feeds (with Google Reader) and read at least the title of every article they publish. I've grouped them into 3 main categories, but otherwise they're in no particular order. Those listed in bold are journals where I have published; they also tend to be the journals most heavily represented in my Mendeley library.

Numerical Analysis and scientific computing:

• BIT Numerical Mathematics
• Journal of Scientific Computing
• SIAM Journal on Scientific Computing (SISC)
• SIAM Journal on Numerical Analysis (SINUM)
• Mathematics of Computation
• Numerische Mathematik
• Applied Numerical Mathematics (APNUM)
• Computational Science and Discovery
• Journal of Computational and Applied Mathematics
• Journal of Computational Physics (JCP)
• Computing in Science and Engineering (CiSE)
• International Journal of Numerical Methods in Fluids (IJNMF)
• IMA Journal of Numerical Analysis
• ACM Transactions on Mathematical Software (TOMS)
• Computer Physics Communications
• Acta Numerica
• math.NA on ArXiv

 

I would add Communications in Computational Physics and Computational Methods in Applied Mathematics, but as far as I know they have no RSS feed.

Nonlinear waves:

• Physica D: Nonlinear Phenomena
• Nonlinearity

 

Here I would add Communications in the Mathematical Sciences, which also has no RSS feed.

General applied math:

• SIAM Review
• SIAM Journal on Applied Math
• IMA Journal of Applied Mathematics

 

Although ArXiv isn't a journal, I've included it here. Indeed, I find useful articles in that feed much more often than for most of the listed journals.

What does your list look like?

I have a book!

I just got notification that my first book, co-authored with Sigal Gottlieb and Chi-Wang Shu, is finally available for pre-order and will ship by the end of the month.  The book is on strong stability preserving methods (the only existing book on the subject) and is, I think, a nice introduction to the subject.  You can read more about it here.  Apparently it has already sold almost 100 copies.  If you do order a copy, you can get a 20% discount until March with the code "DCL032011".