Our LIMS consulting team has gone through a long, evolving process in how we manage our files. To share our experiences with you during this Holiday Season, I thought it appropriate to use a ‘12 Days of Christmas’ analogy where each day can be loosely associated with one year in the timeline of our team. [Our team has only been together for 8 or 9 years, so this will really be the 9 Days of Christmas - or rather the 9 Days of Configuration Management (CM)]

On the First day of CM, my true boss gave to me: a hard drive and a keyboard

In the days of old, most of the team simply stored files on their personal hard drives and these files were shared with others on demand when the need arose. As you might imagine, this lead to many problems with files being lost and other such atrocities.

On the Second day of CM, my true boss gave to me: a shared network drive

In more recent years past, we stored files on shared file systems using simple network drives. This was still inadequate as there was little control over who did what.

On the Third day of CM, my true boss gave to me: Groove

Later, we elected to use peer-to-peer software to improve on issues such as overwriting each other’s files or forgetting to copy our files to the network server. Groove remains to this day one of the most valuable tools our team uses. But Groove is certainly not the ideal tool for maintaining software versions. (wow, a whole paragraph on Groove and not a single snide remark about Microsoft [groove aquisition])

On the Fourth day of CM, my true boss gave to me: Concurrent Versions System

The first quantum leap came in late 2002 when we acquired the first key component in our Source Configuration Management strategy: the version control system CVS. We used CVS exclusively for several years and are still using it today. Although a drastic improvement over shared drives or non-versioning solutions, CVS had some drawbacks. Most annoyingly for our team, CVS did not allow files or folders to be renamed. Further, CVS did a poor job of managing binary files (e.g. Oracle Forms/Reports files, MS Word documents, etc) – every time a binary file is checked in, the entire new version of the file is stored in the repository. It is not uncommon for us to have 50 or more versions of an Oracle Form or Report, each version being 10 megabytes or greater in size. This has lead to high disk usage demands on our repository servers. These and several other downsides led us to search for a solution better than CVS.

On the Seventh day of CM, my true boss gave to me: Subversion

We thought we had seen the star in the East when two team members simultaneously and independently stumbled across a relatively new project that was beginning to gain popularity in the software world. The Subversion project made great effort to proclaim their goal: “to build a version control system that is a compelling replacement for CVS”. We decided to test Subversion out on internal projects before committing to use it for production/customer projects.

Subversion did prove to be an improvement over CVS and addressed many of our gripes with CVS – I even wrote about our anticipated use of Subversion in previous blogs. So we began making plans to convert our customer repositories over to CVS. However, during our investigation phase, we began to realize that while Subversion is great for many types of projects - such as .NET and other web-based projects or Java/C++ projects - it may not be as well suited for Oracle-based, validated LIMS projects.

We went to great lengths to try to squeeze Subversion into the mold of our LIMS projects. But in the end, we could not make it fit. I suppose it is possible that we were stuck in doing something the same way for so long that we could see it no other way. But, I believe we gave our best effort to be open-minded about changing our process to be more closely aligned with the Subversion paradigm, and we still came up short.

The biggest problem we had with Subversion was the way it versions the entire repository instead of individual files. In CVS, we had come to rely on modifying, versioning and delivering single files or groups of files where each file had a precise version. Although Subversion does support the idea of a “version” for each file, the concept is more of a side-effect – versions in Subversion fundamentally apply to the whole repository not to individual files. We could find no reasonable way in Subversion to apply “tags” to individual file versions or to be able to pick and choose precise versions of ad-hoc, individual files when creating builds.

After a long evaluation period we slowly came to the conclusion that Subversion was not the answer for our customer projects. What we really wanted was something that worked like CVS but fixed the things we didn’t like.

On the Ninth day of CM, my true boss gave to me: CVSNT

T’was the night before Christmas and all through the company, not a team member was stirring, not even an intern. Suddenly, Old Saint Jeffrey dropped CVSNT down the chimney. The goal of the CVSNT project, as with Subversion, was to improve upon the flaws of CVS. However, CVSNT differed from Subversion in that it started directly from the open-source CVS codebase and began to add new features that CVS lacked.

CVSNT supports the following features that are not supported by CVS but are important to our LIMS development team:

• File and folder renames
• Efficient storage of binary files
• File level locking and Reserved Edits
• Ability to run the server on Windows
• Simpler setup and configuration of secure, remote access.

We have only been using CVSNT for a few months, but so far it appears to address our concerns with CVS and at the same time is readily compatible with the procedures in place on our LIMS projects. Stated more festively, CVSNT brings all the Holiday cheer of open-source version control without the downsides of CVS which are often associated with the feeling that comes from a night of too much glug.

I hope that this Holiday Season finds you and your LIMS project well.