Version Control – Is It Better to Merge Often or After Completion?

I think the workflow at its core is fine and basically follows the ideas presented here: A successful Git branching model. However, I think the reason it breaks down for you is because you are essentially "off by one" in the merging and testing process:

• Feature branches should be based off of the development branch instead of the stable/master branch. Long-term features are continuously rebased on top of the development branch.
• Feature branches should only be merged into the development branch when they are ready and approved for next release (i.e. when merged, it will definitely go into the next release.)
• When preparing a new release, a release branch is made off the development branch where final testing is done before merged into the stable/master branch for production.

Things definitely gets a little more complicated when a feature depends on another feature not yet merged into development, but as Doc Brown mentioned in his answer, I think "feature toggles" is a good idea here. The nearly completed features are merged into development, but disabled for production use. Dependent features are then rebased on top of development and merged when ready.

The main problem you are going to face is that when you are combining the feature branches to a release branch, you'll need to solve all the inter-branch conflicts. Merge conflicts are the easier ones, since they pop when you are merging specific branches and you can ask the branch owner to solve them(it's far from ideal though, since the branch is not fresh in the owner's memory). But not all conflicts pop us as merge conflicts - some create compilation errors or runtime bugs, and it's not as trivial to figure which feature branches have caused those.

A possible solution can be to shorten the release cycle - adding more rapid "sub-releases", e.g. twice a week. This will limit the number of feature branches you are merging on each sub-release, which in turn limit the conflict potential. This, of course, comes with it's own problems - a frequent release overhead, where the release master needs to choose which features to merge in each sub-release, and after the sub-release the developers need to merge/rebase their pending feature branches(and resolve conflicts).

At any rate, I think your fear of branching-from-develop is unjustified. You are portraying develop as some big playground where all developers push their unfinished scrabbles of untested code - and it's not true. The feature branches fulfill this role. develop, while it might not need to be as stable and as rigorously tested as master, should still have a certain level of stability - the primary rule is not to push to develop if it'll prevent the other developers from continuing to develop even if they merge/rebase develop to their feature branches.

This essentially means that you don't merge a feature branch to develop unless it passes automated tests(doesn't have to be the full suite - if you have a 10-minute suite that catches most bugs and a 5-hour suite that catches even the rarest of bugs, test the feature branches with the 10-minutes suite), so it should be OK to merge it to develop.

Note that master still needs to pass the 5-hour suite, and you have no guarantee a a merged feature branch won't break the 5-hour suite - but neither does your model provide such guarantee. The point is that even if a merged feature branch does break the 5-hour suite - it's still a branch you want in the next release(otherwise you wouldn't have merged it to develop), and the solution is rarely to exclude the feature from the next release.

Update

To answer the asker's first comment to this answer:

When runtime integration bugs arise, the affected feature-set team will be assigned to correct it. If it is caused by code from features created by any of the other teams, fixes are made into pull requests to the offending feature branch. Pull requests are then reviewed by the team that owns that feature, merged in and then merged into the release package. The team that knows how a feature should work makes the fix, the team who owns the offending code reviews it.

This method of solving bugs has several drawbacks compared to solving them as part of the preparation of a feature branch to be merged into develop:

1. The feature-set where the bug happens is usually easy in to figure in both methods. The actual changes that invoked the bug are trivial when branching from develop and very tricky when branching from master. The former only gives you a cue about you about who should be assigned to try solving the bug first, which is not as useful as the actual lead you get from the letter. At any rate, branching from develop allows you to have both hints.

2. The responsibility is backwards. If anything, the owner of the offending code is the one who should fix it, since they know best what they are trying to achieve, and the owner of the feature-set is the one who should review it, because they know best how the different parts of the feature-set should interact with each other.

   But the branch-from-develop approach has an even better way to decide who will be the one to start solving the conflict - it's the one who tries to merge last!

   Now, that claim might seem a little weird and arbitrary - it looks unfair to "punish" the developer who pushed last for being slow. But I believe they are the best choice for starting to solve the problem:

   • They are already in the context of the problem. This is the most important reason - being in context is crucial for solving problems, and entering context is hard. But the developer who pushes last is already in context, because that's the task they are working on. They have already build the mental model that can help them solve this problem.

   • They are available. They don't have something more urgent to do right now, because what they were doing was trying to merge their feature branch, and solving the conflict is required for merging the feature branch.

   • They don't have to actually solve the conflict entirely by themselves - just to be the first ones to look at it. When examining the problem they can decide some other developers need to be involved. Since they are in context, they are in the best position to tell who these other developers are. Also since they are in context, they can help bringing these other developers quickly into context.

3. That pull request into the offending feature branch will be a nightmare. The code in the feature branch works, because the other branch it was conflicting with is not part of it. So, you are sending a fix to a problem that's not yet there, that might have to relies on changes that come with the same code that introduced that problem. There is no sane way to do that without merging/rebasing the other branch(or the new release) into the feature branch - but if you do that you are just using branch-from-develop with develop having it's name replaced on each release.