Node.js – Cache NPM dependencies on Jenkins pipeline

cachingdependenciesJenkinsnode.js

We all know that downloading dependencies with npm can be very time consuming, specially when we are limited to old npm versions.

For me, as a developer, this wasn't such a big deal because I had to do this very few times on my local development machine and everything worked with the node_modules cache in my project's folder. But now I want to take this the applications to a CI environment, with Jenkins.

I realized a huge ammount of time was spent on downloading dependencies with npm. This is a problem because:

  1. npm downloads the dependencies in the project's folder, not a global folder such as Maven's /home/user/.m2

  2. I have to clean up the Jenkins workspace folder in every run to avoid issues with the git checkout.

I want a very elegant solution for caching the npm dependencies on my Jenkins slaves, but so far I can only think of:

  1. Removing everything but the node_modules folders from the Jenkins workspace. I don't like this because I could consume lots of HDD if I keep creating branches for my project. Each branch creates a workspace.

  2. doing something like cp ./node_modules /home/npm_cache after every npm install and then cp /home/npm_cache ./node_modules after the code checkout.

I feel these solutions are terrible. There must be a better way to do this.

Best Answer

What I have done in my Jenkins pipeline for 3 different projects is using tar instead of cp and then npm install instead of npm ci, for each:

  1. cd to your project
  2. npm i
  3. tar cvfz ${HOME}/your_project_node_modules.tar.gz node_modules

Then in the pipeline:

dir(your_project){
  sh "tar xf ${HOME}/your_project_node_modules.tar.gz"
  sh "npm i"
}

Of course it has the disadvantage that with time dependencies change and the install will take longer, but I've managed to reduce my disk space usage in the image by about 0.5GB and tar is much faster then cp (cp ~30 sec, tar ~5 sec)

Total install time went in my case from about 3 minutes to a matter of seconds.

Related Topic