15 practical tips for bioinformaticians.

This is entirely inspired by a blog post of a very similar name from Xianjun Dong on the r-bloggers.com site.  The R-specific focus didn’t do much for me, given that R as a language leaves me annoyed and frustrated, although I do understand why others use it.  I haven’t come across Xianjun’s work before, and have never met him either online or in person, but I hope he doesn’t mind me revisiting his list with a broader scope.  Thanks to Xianjun for creating the original list!

I’ve paraphrased his points in underline, written out my point response, and highlighted what I feel is the take away.  So, lets upgrade the list a bit, shall we?

1. Use a non-random seed.  Actually, that’s pretty good, but the real point should extend this to all areas of your work:  determinism is the key both to debugging and to science – you need to be able to recreate all of your work upon demand.  That’s the basis of how we do science.

2.  The original said set your own tmp directory” so that you don’t overlap toes with other applications.  Frankly, I’d skip that, and instead, suggest you learn how the other applications work!  If you’re running a piece of code, take the time to learn it – and by extension, all of it’s parameters. The biggest mistake I see from novice bioinformaticians is trying to use code they’re not familiar with, and doing something the author never intended.  Don’t just run other people’s tools, use them properly!

3. An R-specific file name hint.  This point was far too R-centric, so I’ll just point you back to another key point: Take the time to learn the biology.  Don’t get so caught up in the programming that you forget that underneath all of the code lies an actual biology or chemistry problem that you’re trying to study, simulate or interpret.  Most often, the best bioinformatics solutions are the ones that are inspired by the biology itself.

4. Create a Readme file for your work. This is actually just the tip of the iceberg – Readme files are the last resort for any serious software project. A reasonable software project should have a wiki or a manual, as well as a host of other documentation. (Bug trackers, feature trackers, unit tests, example data files.)  The list should grow with the size of the project.  If your project is going to last more than a couple of weeks, then a readme file needs to grow into something larger.  Documentation should be an integral part of your coding practice, however you do it.

5. Comment your code.  Yes – please do.  But, don’t just comment your code, write code that doesn’t need comments!  One of the reasons why I love python is because there is a pythonic way to do things, and minimal comments are necessary to make it obvious what its supposed to do.  Of course, anytime you think of a “clever” trick, that’s a prime candidate for extra documentation, and the more clever you are, the more documentation I expect.

6. Backup your code.  Yep – I’m going to agree with the original.  However, I do disagree with the execution.  Don’t just back up your code to an extra disk, get your code into version control.  The only person who doesn’t need version control is the person who never edits their code… and I haven’t met them yet.  If you expect your project to be successful, then expect it to mature over time – and in turn, that you’ll have multiple versions.   Trust me, version control doesn’t just back up, it makes code management and collaboration possible.  Three for the price of one…. or for free if you use github.

7. clean up your intermediate data.  Actually, I think keeping intermediate data around is a useful thing, while you’re working. Yes, biological data can create big files, and you should definitely clean up after yourself, but the more important lesson is to be aware of the resources that are available to you – of which disk space is just one.  Indeed, all of programming is a tradeoff between CPU, Memory and Disk, and they’re interchangeable, of course.  If you’re not aware of the Space-Time tradeoff, then you really haven’t started your journey as a bioinformatician.  Really – this is probably the most important lesson you can learn as a programmer.

8. .bam, not .sam. This point is a bit limited in scope, so lets widen it.  All of the data you’ll ever deal with is going to be in a less-than-optimal format for storage, and it’s on you to figure out what the right format it going to be.  Have VCFs?  Gzip them!  Have .sam files?  Make them .bam files!  Of course, this doesn’t just go for storage: Do the same for how you access them.  That gzipped VCF?  You should have bgzipped it and then tabix indexed it.  Same goes for your Fasta file (FAIDX?), or whatever else you have.  Don’t just use compression, use it to your advantage.

9. Parallelize your code.  Oh man, this is a can of worms.  On the one hand, much of bioinformatics is embarrassingly parallelizeable.  That’s the good news.  The bad news is that threaded/multiprocessed code is harder to debug and maintain.  This should be the last path you go down, after you’ve optimized the heck out of your code.  Don’t parallelize what you can optimize – but use parallelization to overcome resource limitations. And only when you can’t access the resources in any other way.  (If you work with a cluster, though, this may be a quick and dirty way to get more resources…)

10. clean up and back up.  This was just a repeat of earlier points, so lets talk about networking.  The best way to keep yourself current is to listen to what others have to say.  That means making time to go to conferences, reading papers, blogs or even twitter.  Talk to other bioinformaticians because they’ll always have new ideas, and it’s far too easy to get in to a routine where you’re not exposing yourself to whatever is new and exciting.

11. OOP: Inheritance, Encapsulation, Polymorphism. Actually, on this point, I completely agree.  Understanding object oriented programming takes you from being able to write scripts to being able to write a program.  A subtle distinction, but it will broaden your horizons in so many ways, of which the most important is clearly code re-use.  And reusing your existing code means you start developing a toolkit instead of making everything a one off.

12. Save the URL of your references. Again, great start, but don’t just save the URL of your references.  Make notes on everything. Whatever you find useful or inspiring, make a note in your lab book.  Wait, you think bioinformaticians don’t have lab books?  If that’s true, it’s only because you’ve moved on to something else that keeps a permanent record, like version control for your code, or electronic notebooks for your commands.  Make sure everything you do is documented.

13. Keep Learning.  YES!  This!  If you find yourself treading water as a bioinformatician, you’re probably not far from sinking.  Neither programming or biology ever really stand still – there’s always something new that you should get to know.  Keeping up with both fields is tough, but absolutely necessary.

14. Give back what you learn.  Again, gotta agree here.  There are lots of ways to engage the community: share your code, share your experience, share your opinions, share your love of science… but get out and share it somehow.

15. Stand up on occasion.  Ok, I’ll go with this too.  The sitting/standing desks are fantastic, and definitely worth the money, if you can get one.  Bioinformaticians spend way too much time sitting, and you shouldn’t neglect your health.  Or your family, actually.  Don’t forget to work hard, and play hard.

A stab at the future of bioinformatics

I had a conversation the other day about where bioinformatics is headed, and it left me thinking about it for the past few days.  Generally, the question was more about whether bioinformatics (and biotechs) are at the start of something big, or whether this is all a fad.  Unfortunately, I can’t tell the future, but that doesn’t mean I shouldn’t take a guess wild stab in the dark.

Some things are clear because some things never change.  Unless armageddon is upon us or aliens land, we can be sure that sequencing will continue to get cheaper until it hits bottom – by which I mean about the same cost as any other medical test. (At which point, profit margins go up while sequencing costs go down, of course!)  But, that means that for the foreseeable future, we should expect the volume of human sequencing data to continue to rise.

That, naturally, translates pretty directly to an increase in the amount of data that needs to be processed.  Bioinformatics, unlike many other fields, is all about automation and discovery – and in this case, automation is really the big deal.  (I’ll get back to discovery later.)  Pipelines that take care of the human data are going to be more and more valuable, particularly when they add value to the automation and interpretation.  (Obviously, I should disclose that I work for a company that does this.)  I can’t say that I see this need going away any time soon.  However, doing it well requires significant investment and (I’d like to think) skill.  (As an aside, sorry for all of the asides.)

Clearly, though, automation will probably be a big employer of bioinformaticians going forward.  A great pipeline is one that is entirely invisible to the people using it, and keeping a pipeline for the automation of bioinformatics data current isn’t an easy task.  Anyone who has ever said “Great! We’re done building this pipeline!” isn’t on the cutting edge.  Or even on the leading edge.  Or any edge at all.  If you finish a pipeline, it’ll be obsolete before you can commit it to your git repository.

But, the state of the art in any field, bioinformatics included, is all about discovery.  For the most part, I suspect that it means big data.  Sometimes big databases, but definitely big data sets.  (Are you old enough to remember when big data in bioinformatics came in a fasta file, and people thought perl was going to take over the world?)  There are seven billion people on earth, and they all have genomes to be sequenced.  We have so much to discover that every bioinformatician on the planet could work on that full time, and we could keep going for years.

So yes, I’m pretty bullish on the prospects of bioinformaticians in the future.  As long as we perceive knowledge about ourselves is useful, and as long as our own health preoccupies us – for insurance purposes or diagnostics – there will be bioinformatics jobs out there.  (Whether there are too many bioinformaticians is a different story for another post.)  Discovery and re-discovery will really come sharply into focus for the next few decades.

We can figure out some of the more obvious points:

  • Cancer will be a huge driver of sequencing because it changes over time, and so we’ll constantly be driven to sequence again and again looking for markers or subpopulations. It’s a genetic disease and sequencing will give us a window into what it’s doing where nothing else can.  Like physicists and the hunt for subatomic particles, bioinformaticians are going to spend the next hundred years analyzing cancer data sets over and over and over.  There are 3 billion bases in the human genome, and probably as many unique variantions that make a cell oncogenic. (Big time discovery)
  • Rare disease diagnostics should become commonplace.  Can you imagine catching every single childhood disease within two weeks of the birth of a child?  How much suffering would that prevent?   Bioinformaticians will be at the core of that, automating systems to take genetic councillors out of the picture. (discovery turning to automation)
  • Single cell sequencing will eventually become a thing…. and then we’ll have to spend the next decade figuring out how the heck we should interpret it.  That’ll be a whole new field of tools. (discovery!)
  • Integration with medical records will probably happen.  Currently, it’s far from ideal, but mostly because (as far as I can tell) electronic medical records are built for doctors. Bioinformaticians will have to step in and have an impact.  Not that we haven’t seen great strides, but I have yet to hear of an EMR system that handles whole genome sequencing.  (automation.)
  • LIMS.  ugh. It’ll happen and drain the lives from countless bioinformaticians.  No further comment necessary. (automation)

At some point, however, it’s going to become glaringly obvious that the bioinformatics component is the most expensive part of all of the above processes.  Each will drive massive cost savings in healthcare and efficiency, but the actual process of building the tools doesn’t scale the same way as the data generation.

Where does that leave us?  I’d like to think that it’s a bright future for those who are in the field.  Interesting times ahead.

Ants..

This is a strange way to begin, but moving to California has reminded me of an interest in an Algorithm that I’ve always found fascinating: Ant Walks.

I hadn’t expected to return to that particular algorithm, but it turns out there’s a reason why people become fascinated with it. Because it’s somewhat an attempt to describe the behaviour of ants… which California has given me an opportunity to study first hand.

I’m moving in a week or two, but I have to admit, I have a love/hate relationship with the Ant colony in the back yard. I won’t really miss them, because they’re seriously everywhere. Although I’ve learned how to keep them out of the house, and they dont’ really bother me much, they’re persistent and highly effective at finding food. Especially crumbs left on the kitchen floor. (By the way, strategic placement of ant repellent, and the ants actually have a pretty hard time finding their way in… but that’s another post for another day.)

Regardless, the few times that the ants have found their way inside have inspired me to watch them and learn a bit about how they do what they do – and it’s remarkably similar to the algorithm based off of their behaviour. First, they take advantage of sheer numbers. They don’t really care about any one individual, and thus they just send each ant out to wander around. Basically, it’s just a divide and conquer, with zero planning. The more ants they send out, the more likely they are to find something. If you had only two or three ants, it would be futile… but 50-100 ants all wandering in a room with a small number of crumbs will result in the crumbs all being found.

And then there’s the whole thing about the trails. Watching them run back and forth along the trails really shows you that the ants do know exactly where they’re going, when they have somewhere to be. When they get to the end, they seem to go back into the “seeking” mode, so you can concentrate the search for relevance to a smaller area, for a more directed random search.

All and all, it’s fascinating. Unfortunately, unlike Richard Feynman, I haven’t had the time to set up Ant Ferries as a method of discouraging the ants from returning – my daughter and wife are patient, but not THAT patient – but that doesn’t mean I haven’t had a chance to observe them.  I have to admit, of all the things that I thought would entertain me in  California, I didn’t expect that Ants would be on that list.

Anyone interested in doing some topology experiments? (-;