I had another topic queued up this morning, but an email from my sister-in-law reminded me of a more pressing beef: Lack of collaboration in the sciences. And, of course, I have no statistics to back this up, so I’m going to put this out there and see if anyone has anything to comment on the topic.
My contention is that the current methods for funding scientists is the culprit for driving less efficient science, mixed with a healthy dose of Zero Sum Game thinking.
First, my biggest pet peeve is that scientists – and bioinformaticians in particular – spend a lot of time reinventing the wheel. How many SNP callers are currently available? How many ChiP-Seq packages? How many aligners? And, more importantly, how can you tell one from the other? (How many of the hundreds of snp callers have you actually used?)
It’s a pretty annoying aspect of bioinformatics that people seem to feel the need to start from scratch on a new project every time they say “I could tweak a parameter in this alignment algorithm…” and then off they go, writing aligner #23,483,337 from scratch instead of modifying the existing aligner. At some point, we’ll have more aligners than genomes! (Ok, that’s a shameless hyperbole.)
But, the point stands. Bioinformaticians create a plethora of software that solve problems that are not entirely new. While I’m not saying that bioinformaticians are working on solved problems, I am asserting that the creation of novel software packages is an inefficient way to tackle problems that someone else has already invested time/money into building software for. But I’ll come back to that in a minute.
But why is the default behavior to write your own package instead of building on top of an existing one? Well, that’s clear: Publications. In science, the method of determining your progress is how many journal publications you have, skewed by some “impact factor” for how impressive the name of the journal is. The problem is that this is a terrible metric to judge progress and contribution. Solving a difficult problem in an existing piece of software doesn’t merit a publication, but wasting 4 months to rewrite a piece of software DOES.
The science community, in general, and the funding community more specifically, will reward you for doing wasteful work instead of focusing your energies where it’s needed. This tends to squash software collaborations before they can take off simply by encouraging a proliferation of useless software that is rewarded because it’s novel.
There are examples of bioinformatics packages where collaboration is a bit more encouraged – and those provide models for more efficient ways of doing research. For instance, in the molecular dynamics community, Charmm and Amber are the two software frameworks around which most people have gathered. Grad students don’t start their degree by being told to re-write one or the other packages, but are instead told to learn one and then add modules to it. Eventually the modules are released along with a publication describing the model. (Or left to rot in a dingy hard drive somewhere if they’re not useful.) Publications come from the work done and the algorithm modifications being explained. That, to me, seems like a better model – and means everyone doesn’t start from scratch
If you’re wondering where I’m going with this, it’s not towards the Microsoft model where everyone does bioinformatics in Excel, using Microsoft generated code.
Instead, I’d like to propose a coordinated bioinformatics code-base. Not a single package, but a unified set of hooks instead. Imagine one code base, where you could write a module and add it to a big git hub of bioinformatics code – and re-use a common (well debugged) core set of functions that handle many of the common pieces. You could swap out aligner implementations and have modular common output formats. You could build a chip-seq engine, and use modular functions for FDR calculations, replacing them as needed. Imagine you could collaborate on code design with someone else – and when you’re done, you get a proper paper on the algorithm, not an application note announcing yet another package.
(We have been better in the past couple years with tool sets like SAMTools, but that deals with a single common file format. Imagine if that also allowed for much bigger projects like providing core functions for RNA-Seq or CNV analysis… but I digress.)
Even better, if we all developed around a single set of common hooks, you can imagine that, at the end of the day (once you’ve submitted your repository to the main trunk), someone like the Galaxy team would simply vacuum up your modules and instantly make your code available to every bioinformatician and biologist out there. Instant usability!
While this model of bioinformatics development would take a small team of core maintainers for the common core and hooks, much the same way Linux has Linus Torvalds working on the Kernel, it would also cut down severely on code duplication, bugs in bioinformatics code and the plethora of software packages that never get used.
I don’t think this is an unachievable goal, either for the DIY bioinformatics community, the Open Source bioinformatics community or the academic bioinformatics community. Indeed, if all three of those decided to work together, it could be a very powerful movement. Moreso, corporate bioinformatics could be a strong player in it, providing support and development for users, much the way corporate Linux players have done for the past two decades.
What is needed, however, is buy-in from some influential people, and some influential labs. Putting aside their own home grown software and investing in a common core is probably a challenging concept, but it could be done – and the rewards would be dramatic.
Finally, coming back to the funding issue. Agencies funding bioinformatics work would also save a lot of money by investing in this type of framework. It would ensure more time is spent on more useful coding, more time is spent on publications that do more to describe algorithms and to ensure higher quality code is being produced at the end of the day. The big difference is that they’d have to start accepting that bioinformatics papers shouldn’t be about “new software” available, but “new statistics”, “new algorithms” and “new methods” – which may require a paradigm change in the way we evaluate bioinformatics funding.
Anyhow, I can always dream.
Notes: Yes, there are software frameworks out there that could be used to get the ball rolling. I know Galaxy does have some fantastic tools, but (if I’m not mistaken), it doesn’t provide a common framework for coding – only for interacting with the software. I’m also aware that Charmm and Amber have problems – mainly because they were developed by competing labs that failed to become entirely enclusive of the community, or to invest substantially in maintaining the infrastructure in a clean way.Finally, Yes, the licensing of this code would determine the extent of corporate participation, but the GPL provides at least one successful example of this working.