Java – How to reduce the build time from once a day to once every 15 minutes

buildsdesignjava

I have a large codebase of Java/JSP files that needs to be compiled and checked for errors. The compilation time is long enough that we only run it once a day. However, I'd like it to be more frequent than that, say every 15 minute interval.

But I'm not able to figure out how to minimize the build time to allow this. Running the ANT script, downloading all files off Version Control, compiling JSP's into JAVAs and then compiling those JAVAs into Class files takes an hour and a half.

My first solution was to download a copy of the codebase and sync changes from the server (svn update) every 15 minutes, in order to reduce the downloading time. However, I have not been able to minimize the compilation time. I'm using Jasper to compile the JSPs and building all the files (into JAVA files) takes several minutes. Compiling the produced Java files takes another 15 minutes. To tackle this, I though about individually compiling the "updated" JSP files and copying them back to the JAVA source. This would take away the JSP->JAVA stage. However, ANT and Jasper haven't made it easy to compile individual files and also ensure that no new errors show up.

Should I settle for once a day build? Or is there a better solution that can be applied here?

Best Answer

I did this type of thing at a previous company. We had a very large C application that took ages to do a complete build. 45 minute compile time, but we managed to get it down to less than 6.

Measure:

Firstly as you are doing I would measure each part of the process:

  1. Check out 15 minutes.
  2. Compiling Java branch takes 20
  3. Compiling JSP branch takes 40

Time: 75 minutes

However you state in the question that is takes an hour and a half so we have a missing 15 minutes, where is this is the build process ?

Running the ANT script, downloading all files off Version Control, compiling JSP's into JAVAs and then compiling those JAVAs into Class files takes an hour and a half.

So the first thing is to do an update of SVN rather than a checkout as in comments you suggest, this reduces the build time to 62 minutes. (Average 6 builds a day)

Next I would think look at the build scripts, is it possible to build your JAVA code or JSP in parallel. As it's legacy code it might not have been told to build over multiple CPU's. Our project was still sitting on a single CPU. You mention ANT it has the ability to build in parallel. Parallel Tasks

If it already doing this does writing them to disk pose an issue ? I ask this because if you are creating a lot of objects, the disk could be hitting it's bandwidth. Try an SSD drive for building. I mention this because it's a low cost solution to implement.

Other things to consider: You want fast feedback on a change, does it need to be full feedback ?

  1. Linting the code for faster feedback for some errors.
  2. Unit tests.
  3. CI server to distribute the load over several machines.

Long Term:

  1. Split the code out into multiple parts, do you have utilities that could be compiled independently, then included later ?
  2. PMD can show you where duplicate code is being used. Legacy projects normally have this issue where the same code is included in many places.
Related Topic