Scala sbt: Multiple dependencies in sbt

Summary of important behavior differences:

• dependencies are installed on both:

  • npm install from a directory that contains package.json
  • npm install $package on any other directory

• devDependencies are:

  • also installed on npm install on a directory that contains package.json, unless you pass the --production flag (go upvote Gayan Charith's answer).
  • not installed on npm install "$package" on any other directory, unless you give it the --dev option.
  • are not installed transitively.

• peerDependencies:

  • before 3.0: are always installed if missing, and raise an error if multiple incompatible versions of the dependency would be used by different dependencies.
  • expected to start on 3.0 (untested): give a warning if missing on npm install, and you have to solve the dependency yourself manually. When running, if the dependency is missing, you get an error (mentioned by @nextgentech) This explains it nicely: https://flaviocopes.com/npm-peer-dependencies/
  • in version 7 peerDependencies are automatically installed unless an upstream dependency conflict is present that cannot be automatically resolved

• Transitivity (mentioned by Ben Hutchison):

  • dependencies are installed transitively: if A requires B, and B requires C, then C gets installed, otherwise, B could not work, and neither would A.

  • devDependencies is not installed transitively. E.g. we don't need to test B to test A, so B's testing dependencies can be left out.

Related options not discussed here:

• bundledDependencies which is discussed on the following question: Advantages of bundledDependencies over normal dependencies in npm
• optionalDependencies (mentioned by Aidan Feldman)

devDependencies

dependencies are required to run, devDependencies only to develop, e.g.: unit tests, CoffeeScript to JavaScript transpilation, minification, ...

If you are going to develop a package, you download it (e.g. via git clone), go to its root which contains package.json, and run:

npm install

Since you have the actual source, it is clear that you want to develop it, so by default, both dependencies (since you must, of course, run to develop) and devDependency dependencies are also installed.

If however, you are only an end user who just wants to install a package to use it, you will do from any directory:

npm install "$package"

In that case, you normally don't want the development dependencies, so you just get what is needed to use the package: dependencies.

If you really want to install development packages in that case, you can set the dev configuration option to true, possibly from the command line as:

npm install "$package" --dev

The option is false by default since this is a much less common case.

peerDependencies

(Tested before 3.0)

Source: https://nodejs.org/en/blog/npm/peer-dependencies/

With regular dependencies, you can have multiple versions of the dependency: it's simply installed inside the node_modules of the dependency.

E.g. if dependency1 and dependency2 both depend on dependency3 at different versions the project tree will look like:

root/node_modules/
                 |
                 +- dependency1/node_modules/
                 |                          |
                 |                          +- dependency3 v1.0/
                 |
                 |
                 +- dependency2/node_modules/
                                            |
                                            +- dependency3 v2.0/

Plugins, however, are packages that normally don't require the other package, which is called the host in this context. Instead:

• plugins are required by the host
• plugins offer a standard interface that the host expects to find
• only the host will be called directly by the user, so there must be a single version of it.

E.g. if dependency1 and dependency2 peer depend on dependency3, the project tree will look like:

root/node_modules/
                 |
                 +- dependency1/
                 |
                 +- dependency2/
                 |
                 +- dependency3 v1.0/

This happens even though you never mention dependency3 in your package.json file.

I think this is an instance of the Inversion of Control design pattern.

A prototypical example of peer dependencies is Grunt, the host, and its plugins.

For example, on a Grunt plugin like https://github.com/gruntjs/grunt-contrib-uglify, you will see that:

• grunt is a peer-dependency
• the only require('grunt') is under tests/: it's not actually used by the program.

Then, when the user will use a plugin, he will implicitly require the plugin from the Gruntfile by adding a grunt.loadNpmTasks('grunt-contrib-uglify') line, but it's grunt that the user will call directly.

This would not work then if each plugin required a different Grunt version.

Manual

I think the documentation answers the question quite well, maybe you are just not familiar enough with node / other package managers. I probably only understand it because I know a bit about Ruby bundler.

The key line is:

These things will be installed when doing npm link or npm install from the root of a package and can be managed like any other npm configuration parameter. See npm-config(7) for more on the topic.

And then under npm-config(7) find dev:

Default: false
Type: Boolean

Install dev-dependencies along with packages.

The way to exclude specific transitive dependency is this:

1. Run sbt evicted to figure out which of the project dependencies is pulling in outdated library, let's assume the problematic library is: com.typesafe.slick.

2. Add the following exclude (the parentheses are important):

   ("com.typesafe" %% "slick" % "3.1.1").exclude("org.scala-lang.modules", "scala-xml_2.11")

   Add this normally as you would be listing dependencies.

3. This will prevent sbt from including any version of scala-xml that was coming in as a transitive dependency of Slick.