NodePy version 0.4 released

NodePy is a Python package for analyzing numerical integrators for initial value ODEs.  It's essentially a collection of all the kinds of analysis I've used in my time integrator research, collected in a single object-oriented package.  

If you have a new Runge-Kutta method and want to know all about it, NodePy can tell you most anything.  If you want to design new time integration methods, NodePy can help you.

Although I'm rather proud of it, it fills a very small niche in the world and I'm not aware of anyone using it outside of my group and close collaborators.  If you've used it, please let me know in the comments.

One of the thorniest issues in NodePy previously was that floating-point representations of method coefficients were sometimes insufficient, especially when studying very high order methods.  I've now updated NodePy to use Sympy Rationals (and radicals, etc.) wherever possible, allowing exact analysis of many properties.

That and much more awaits in NodePy version 0.4, now available via pip.

7 Habits of the Open Scientist: #2 -- Reproducible Research

Note: this post is part of a series on habits of the open scientist.  Here I discuss the second habit, reproducible research.  The previous post was on open scientific publishing.

Reproducible research

Reproducibility is part of the definition of science: if the results of your experiment cannot be replicated by different people in a different location, then you're not doing science.  Far from being a mere philosophic concern, reproducible research has been a key issue in prominent controversies like climategate and cancer research clinical trials.

Especially disconcerting is the typical irreproducibility of scientific work involving computer code:

“Computational science is facing a credibility crisis: it’s impossible to verify most of the computational results presented at conferences and in papers today.” (LeVeque, Mitchell, Stodden, CiSE 2012)

Frankly, I used to find that I was often unable to reproduce my own computational results after a few months, because I had not maintained sufficiently detailed notes about my code and my computing environment.

The open scientist ensures that the entire research compendium -- including not only the paper but the data, source code, parameters, post-processing, and computing environment -- is made freely available, preferably in a way that facilitates its reuse by others.

I won't spend more time motivating reproducible research, since others have done that much better than I could.  Instead, let me focus on the relatively easy first steps you can take to make your research more reproducible.

The bare minimum: publish your code and data

If you wish to set an example of good reproducible computational research practices, I have good news for you: the bar is very low at the moment.  The reason why "it's impossible to verify most of the computational results" is that most researchers don't release their code and data.  The first step toward working reproducibly is simply to put the code and data that is used in your published research out in the open.

If you don't want to release your code to the public, please read about why you should and why you can.  Once you're convinced, go endorse the Science Code Manifesto.

Releasing your code and data can be as simple as posting a tarball on your website with a reference to the paper it pertains to.  Or you may wish to start putting all your code out in the open on Bitbucket or Github, like I do.  I don't claim that these are the best solutions possible, but they are a big step forward from keeping everything on your own hard drive.

When you release your code and data, it is important to use an appropriate license.  Victoria Stodden, a leader in the reproducible research movement, recommends the use of a permissive license like modified BSD for code and Science Commons Database Protocol for data.  Together with the Creative Commons BY license for media (that I mentioned in my last post), these comprise the Reproducible Research Standard, a convenient amalgamation of licenses for open science.

Be sure to include a mention of reproducibility in your paper, along with links to the code and data.  If you release your work under the RRS, I suggest using this citation.

Real benefits

The open scientist may adopt reproducible research practices for philosophical reasons, but he soon finds that they bring more direct benefits.  Because he writes code and prepares data with the expectation that it will be seen by others, the open scientist finds it much easier for himself, students, and colleagues to build on past work.  New collaborations are formed when others discover his work through openly released code and data.  And (as in the case of this paper, for example) the code itself may be the main subject of publications in journals that have come to recognize the importance of scientific software.

Taking it further

Like free and open scientific publishing, reproducible research has become a very large movement, and only a book could hope to cover it all.  Here I've merely distilled some basic practical suggestions.

Openly releasing code and data is only the first step.  Open scientists may wish to adopt tools that track code provenance and ensure a fully reproducible workflow, such as

• Madagascar
• VisTrails
• knitr
• Sumatra
• VCR

Next up: Habit #3 -- Pre-publication dissemination of research.

7 Habits of the Open Scientist

Science has always been based on a fundamental culture of openness.  The scientific community rewards individuals for sharing their discoveries through perpetual attribution, and the community benefits by through the ability to build on discoveries made by individuals.  Furthermore, scientific discoveries are not generally accepted until they have been verified or reproduced independently, which requires open communication.

 

Historically, openness simply meant publishing one's methods and results in the scientific literature.  This enabled scientists all over the world to learn about essential advances made by their colleagues, modulo a few barriers.  One needed to have access to expensive library collections, to spend substantial time and effort searching the literature, and to wait while research conducted by other groups was refereed, published, and distributed.

 

Nowadays it is possible to practice a fundamentally more open kind of research -- one in which we have immediate, free, indexed, universal access to scientific discoveries.  The new vision of open science is painted in lucid tones in Michael Nielsen's Reinventing Discovery.  After reading Nielsen's book, I was hungry to begin practicing open science, but not exactly sure where to start.  Here are seven ways I'm aware of.  Each will be the subject of a longer forthcoming post.

 

I believe that every scientist has a moral imperative to adopt the first two:

 

1. Freely accessible publications.  At a minimum, make sure that everyone is allowed to read your research.

2. Reproducible research.  Release your code and data so  that anyone who wants to can verify or build directly on your work.

 

The remaining five are marks of a truly open scientist:

 

3. Pre-publication dissemination of research.  Just because peer-review and journals take time, that doesn't mean you need to embargo your audience.

4. Open collaboration through social media.  Find the person who knows that one thing you need, through new scientific networking tools -- and share your own expertise where it's needed most.

5. Live open science.  Tell people about your marvelous discoveries -- as you make them.

6. Open expository writing.  Teach others about the field you work in through a blog or online book.

7.  Open bibliographies and reviews.  Let your colleagues know what you're reading, and what you've learned from it.

Endorse the Science Code Manifesto

Read it.
Sign it.
Do it.
http://sciencecodemanifesto.org.

nodepy 0.3 available via easy_install

To facilitate my research and perhaps help someone else out there, I develop a python package based around numerical ODE solvers (Runge-Kutta methods, multistep methods, etc.) as objects. The package is called nodepy, and has somewhat limited functionality. However, it contains a very nice implementation of rooted trees, including the ability to compute all the things necessary for deriving order conditions of general linear methods. It also has a lot of nice functionality for Runge-Kutta methods, including a lot of things related to low-storage methods and embedded methods.

As of today, the package is finally available on the PyPI server, and therefore can be installed using

easy_install nodepy

Hopefully this will encourage interested parties to try it out (or better yet, to contribute!)