Continuous Integration / Deployment: Test on commit, pull request, or what

First of all, the use of feature branches (to isolate the work done on a feature) and CI (to find integration problems as soon as they are committed) are slightly at odds.

In my opinion, running CI on feature branches is a waste of time. As feature branches come and go frequently, the CI tooling would have to be reconfigured over and over again. And that for a branch that most likely only gets updates from one or two sources that coordinate their check-ins to avoid the problems a CI system is meant to detect.
Thus, there is also no point in having the feature branches on the master repository server.

As for questions 1 and 3: It is the developer's responsibility to ensure the build on the main development branch does not break when they merge their feature-branch into it. How they do that is their problem, but two possible ways are:

• Pull changes made to the main development branch into the feature branch on a regular basis (e.g. daily)
• When the feature is done, merge the main development branch into the feature branch and push the merge result onto the main development branch.

In either case, the obvious integration issues (e.g. renamed classes/files) are found and fixed first on the feature branch. The more subtle issues are most likely only found when the nightly build runs and should be fixed there and then.

Wikipedia gives pretty good summaries of most of these terms. Here is my take on them:

• Build automation is automating how the software is built instead of manually invoking the compiler. This would be accomplished via tools such as e.g. Make or Ant.

• Deployment automation is taking your built software and deploying or installing it on a test or production system.

• Continuous integration means having an automated process build your software continuously as developers check in code, and run unit tests to ensure the code still works. For example, every 15 to 30 minutes a server might wake up, scan VCS for new check-ins, then update and build the project if any changes were made. In addition to performing compile steps, this is also a great opportunity to run automated unit tests and code quality checks.

• Continuous delivery is a combination of all of the previous concepts where the software builds are also deployed to a test system, optionally with tests performed and reports generated.

At the very least, you need to have build automation, i.e. a build script of some sort. That allows you to click one button or issue one command to build your project. The benefit to this is reducing errors from manually running steps. Complex build environments might involve generating code (think DAOs from configs, interface code such as JAXB), compiling code, packaging it up, customizing metadata, etc. With a lot of stuff to do you need a checklist: why not make the checklist be your build script, and use a tool to run it? It reduces errors and provides consistency.

Next up is CI: this is really good to have but not strictly required. It helps identify build problems early. If you have multiple developers checking in code throughout the day and perhaps not syncing up their own workspaces constantly, there is a risk that their changes will interfere with each other. I am referring specifically to static code errors, not version control conflicts. A CI build server will mitigate this risk.

Finally we have the deployment steps. The idea here is to save time and reduce error from manually deploying software. Much like build automation, there are a hundred ways to screw up a software deployment. I have personally stayed late at the office to fix manual deployment problems on many occasions when we need a functioning system for customers coming on-site tomorrow. Automating multiple systems introduces more risk: instead of one system possibly crashing or having weird errors, we now have multiple systems that can go wrong. However, that risk is far lower than somebody missing a step on a checklist or issuing the wrong command and messing up a deployment. If you are lucky you can simply restore a DB backup and start over, if you are unlucky an error might cause the system to function incorrectly. Is it a software defect? Did the technician not set a configuration correctly? This takes time to diagnose, time that you may not have and time that need not be spent if you automate the process.