Branching and versioning strategy for shared libraries

development-processreleaseversion controlversioning

These posts seem related, but my brain is starting to melt, trying to think this through 😛

My employer has just started using source control, primarily because before they hired more developers, the "repository" was the hard drive of the lone dev, who works mainly from home. All of the .NET code he'd written was checked in en masse, and there is a lot of duplicated (read: copy-pasted) functionality. Right now, our SCM system is glorified backup.

I'd like to pull some of the duplicated code into shared libraries. I'm leaving the original repo alone so that we don't break anything—we can move and/or refactor existing code as and when needed. So I've set up a repo just for new code, including libraries.

My problem revolves around versioning the libraries without miring us in excessive process: having acknowledged that we need a more coherent approach, with more devs all writing very similar code, management and the other devs are open to reorganising things, but it probably won't go down well if the solution starts to affect productivity.

The ideal solution in my obsessive mind is to build the libraries separately, with each dependent project built against deliberately chosen, compatible versions of them. That way, we know exactly which clients have which versions of which libraries, can more reliably reproduce bugs, maintain independent release branches for products and libraries, and not break each other's projects when changing shared code.

This makes updating the libraries cumbersome though, particularly for the dev who works from home. I expect the libraries to change quickly, at least initially as we (finally) pull the common bits together. It's quite possible that I'm completely overthinking this and we'd be fine just building everything against the most recent library commits, but I'd like to at least be prepared for the day we decide that some components have to be independently versioned and distributed. The fact that some of the libraries will have to be installed in the GAC makes versioning particularly important.

So my question is: what am I missing? I feel like I've fixated on one solution and am now trying to find variations on it that will make changes smoother. What kinds of strategies have you used to solve this kind of problem before? I realise this question is all over the place; I'll do my best to clean up and clarify any points of uncertainty.

And as much as I'd love to use Mercurial, we've already spent money on a centralised commercial SCM (Vault) and switching isn't an option. Besides, I think the issue here goes deeper than choice of version control tool.

Best Answer

I commend your initiative. This should yield a number of benefits for the organization as you implement this. I would move the code rather than copy it. Having copies will likely result in incompatible changes which will need to be resolved.

There will be some pain as the libraries are developed and stabilized. Once that is done, the benefits will arrive. Remember that the interfaces to the library are essentially a contract with the projects working with the contract. You may have the advantage of removing old interfaces as you may be able to determine if they are used.

While the libraries are stabilizing, getting new libraries should probably be part of the code update process. You may want to schedule commits of library changes. Announcing the code being moved may help reduce conflicting changes.

The libraries should be treated as separate projects and stabilized as soon as possible. Once they are stabilized (particularly the interfaces) it should be easier to integrate changes with other projects. New libraries should work with old code. Tag stable library releases with their own release id. Try to treat the libraries as you would third party libraries.

Related Solutions

Release Management – Version Control Best Practices

From what I see, you are referring to configuration management, however, if there are different parts to your system and you want to put them in version control to keep them consistent, the best I can think of is:

A) To maintain a file (in XML, JSON or whatever serialization format you like) where you describe what version of each thing you need for each component.

{
  "JAVA": ">1.7.0_05",
  "MYSQL": ">=5.0.28",
  ...
}

Have each component import this file into constants and assert their values against the environment they are running on, there are ways to get this information from wherever your application is deployed. If they don't match the desired values (or value ranges even), throw a warning.

B) If you have different components and you use something like Mercurial for version control, then you could use something like the subrepository feature, where your top/manager repository essentially does configuration management, this is, the top repository will keep track what versions of each component are being used across in a given moment in time. Picture this:

project_x #250 <- top repository at changeset 250
|--component_a #1232 <-component a at changeset 1232
|--component_b #23 <- component b at changeset 23
|--component_c #32 <- component c at changeset 32

And then with component_a's new version (say 1233) you need a new version of component_c, but not component_b, so you update component_a and component_c and commit project_x to a new version that records those changes

project_x #251 <- top repository at changeset 251, recording 1233 and 33 for a and c
|--component_a #1233 <-component a now at at changeset 1233
|--component_b #23 <- component b at changeset 23
|--component_c #33 <- component c now at changeset 33

Keep in mind that it might not be the best idea to store java or mysql itself in version control, but as mentioned earlier, this would be effective configuration management for project-independent components.

If you want to keep track of Java, MySQL, etc, "A)" provides a good mechanism to assert that configuration.

The "project_x" repo can have a development branch and a production branch where it tracks different component configurations.

In summary

Have a manifest to which you can code against and assert your configuration, and for your project independant components you can use Mercurial subrepos (or git submodules) to keep track of the version configuration among them.

Branching Strategy for Test Environment

Your branching strategy looks really good to me. I have done the same strategy in the past and it works fine. Draw it up on a whiteboard and get all your devs to understand it so that people do the right work in the right branch. Teach and explain to everyone the switch command and get everyone to doublecheck the branch that they are working on. (Or alternatively just check out the entire repo... depending on your code size :) Remember... svn revert is your best friend!

Personally I prefer one person to be the "merge/branch" person (with a few backup people as reserves) to ensure that everything is kept under control and consistent. Let that person become your SVN guru and you'll be away.

A few other helpful hints:

Encourage frequent SVN updates and SVN commits. Every day is preferable.
Cross branch merges should also be done every day, or alternatively whenever a bug is fixed. Do them early and do them often! (you'll get good at it real quick).
Get a good diff tool - beyondcompare is ace. The standard tortoiseSVN one... not too good.
Don't check in stuff that changes upon compilation (like your output directory)
Try to clean up your repo before you start branching (get rid of files that don't need to be under version control - things like external libraries etc). The smaller your repo, the better
Changes to your Production branch and QA branches should be as small and short as possible - don't start refactoring code there, just fix the bug.
Make sure you branch from the top level of your solution - and if you have a DB I hope you've scripted all of your DB stuff (like stored procs or triggers)

Also tell people not to move folders around unless it's strictly necessary. This will make your merging much easier :) (Don't do what I did, launch upon a massive directory restructure halfway through a huge change to trunk which screwed up all of our merges... I was pretty popular after that).

Best Answer

Related Solutions

Release Management – Version Control Best Practices

In summary

Branching Strategy for Test Environment

Related Topic