Jenkins – How to make a Jenkins job start after multiple simultaneous upstream jobs succeed

buildcontinuous integrationJenkinspipelineplugins

In order to get the fastest feedback possible, we occasionally want Jenkins jobs to run in Parallel. Jenkins has the ability to start multiple downstream jobs (or 'fork' the pipeline) when a job finishes. However, Jenkins doesn't seem to have any way of making a downstream job only start of all branches of that fork succeed (or 'joining' the fork back together).

Jenkins has a "Build after other projects are built" button, but I interpret that as "start this job when any upstream job finishes" (not "start this job when all upstream jobs succeed").

Here is a visualization of what I'm talking about. Does anyone know if a plugin exists to do what I'm after?
Build Pipeline

Edit:

When I originally posted this question in 2012, Jason's answer (the Join and Promoted Build plugins) was the best, and the solution I went with.

However, dnozay's answer (The Build Flow plugin) was made popular a year or so after this question, which is a much better answer. For what it's worth, if people ask me this question today, I now recommend that instead.

Best Answer

Pipeline plugin

You can use the Pipeline Plugin (formerly workflow-plugin).

It comes with many examples, and you can follow this tutorial.

e.g.

// build
stage 'build'
...

// deploy
stage 'deploy'
...

// run tests in parallel
stage 'test'
parallel 'functional': {
  ...
}, 'performance': {
  ...
}

// promote artifacts
stage 'promote'
...

Build flow plugin

You can also use the Build Flow Plugin. It is simply awesome - but it is deprecated (development frozen).

Setting up the jobs

Create jobs for:

  • build
  • deploy
  • performance tests
  • functional tests
  • promotion

Setting up the upstream

  1. in the upstream (here build) create a unique artifact, e.g.:

    echo ${BUILD_TAG} > build.tag

  2. archive the build.tag artifact.

  3. record fingerprints to track file usage; if any job copies the same build.tag file and records fingerprints, you will be able to track the parent.
  4. Configure to get promoted when promotion job is successful.

Setting up the downstream jobs

  1. I use 2 parameters PARENT_JOB_NAME and PARENT_BUILD_NUMBER

  2. Copy the artifacts from upstream build job using the Copy Artifact Plugin

    • Project name = ${PARENT_JOB_NAME}
    • Which build = ${PARENT_BUILD_NUMBER}
    • Artifacts to copy = build.tag

  3. Record fingerprints; that's crucial.

Setting up the downstream promotion job

Do the same as the above, to establish upstream-downstream relationship. It does not need any build step. You can perform additional post-build actions like "hey QA, it's your turn".

Create a build flow job

// start with the build
parent = build("build")
parent_job_name = parent.environment["JOB_NAME"]
parent_build_number = parent.environment["BUILD_NUMBER"]

// then deploy
build("deploy")

// then your qualifying tests
parallel (
    { build("functional tests",
          PARENT_BUILD_NUMBER: parent_build_number,
          PARENT_JOB_NAME: parent_job_name) },
    { build("performance tests",
          PARENT_BUILD_NUMBER: parent_build_number,
          PARENT_JOB_NAME: parent_job_name) }
)

// if nothing failed till now...
build("promotion",
    PARENT_BUILD_NUMBER: parent_build_number,
    PARENT_JOB_NAME: parent_job_name)

// knock yourself out...
build("more expensive QA tests",
    PARENT_BUILD_NUMBER: parent_build_number,
    PARENT_JOB_NAME: parent_job_name)

good luck.

Related Topic