Best practice for multi-module design local dependencies

node.js

This question may have been asked before and if so feel free to link to the answer. I'm looking for a modern and effective recipe for a common problem. I have a project with a main module, A, that depends on another module, B, that I develop separately. this module has 2-3 (C-E) other modules it depends on, each of which belong to me as well. Some feel comfortable locating all modules in one svn or Git repo. My experience and gut tell me each module belongs in and individual repo. I currently face this dilemma with a NodeJS/Electron project I'm working on but the problem/solution is certainly technology agnostic. While I don't know enough about local modules in node I am more than familiar with the concepts. I'm just struggling on how to best apply what I've learned to this project I'm currently on.

I'm going to iterate the pros and cons of both approaches below as I've seen from my 17+ years of experience. I'm interested in how others have addressed this issue and is totally separate modules the way to go in this day and age? Are there any new tricks to managing local dependencies that I've missed over the years?

All in one repo

Pros:

  • Everything can be compiled/debugged easily as one complete program in the classic way of thinking.
  • Changes made in the individual modules are immediately visible and represented when launching the main module. Everything is "linked" together.
  • Lower barrier to entry for junior-mid level engineers.

Cons:

  • Project cohesion suffers as boundary lines of responsibilities blur. (Too easy to link code from defendants to dependencies & vice-versa.)
  • Slower more complicated builds as additional logic must be factored in to deal with compiling each module in the proper order and in the proper way.

Separate repos

Pros:

  • Better cohesion. Lines of responsibility are clearly delineated
  • Instant/immediate re-usability. (In order to prove the module works you must test it outside of its dependent module which enforces reusability from inception.)
  • Smaller simpler manageable builds. The dependencies are built once per release and the main project only needs to declare its dependencies rather than build each of them.

Cons:

  • Additional complexity iterating changes in the module dependencies. A dependency must be built & released (or semi-released) before being run in the main module.
  • Add'l complexities in debugging. You can't always step into the dependency source with this approach depending on the technology stack and approach of assembling the overall app.
  • Requires more discipline, higher barrier to entry for junior-mid level engineers.

I mentioned the inability to step into code while debugging as a con but this does not apply as much to JavaScript where all source is always visible. However I still face challenges setting up debugging in my IDE when the modules are externalized.

Best Answer

My two cents:

Use separate repositories on shared projects.

In other words, if a project is used by more than one other project (or intended to be used in multiple projects, i.e. a library), it goes in its own repository.

We are suffering this pain where I currently work. Here's what the current conversation sounds like:

ME: Shouldn't projects build straight from the source code repository, as a matter of policy?

SHE: Yes, they should. But we don't have a working CI server, and you can't expect to enforce this rule without one.

ME: Don't we have one of those?

SHE: Yes, but it's on a cloud server which doesn't have access to the shared drive where we keep the latest build binaries.

ME: Then why don't we just get the CI server to build those binaries for us?

SHE: Really, we should be taking project references instead of binary references. That way, we can be sure that each project has everything it needs to build.

ME: If you make me do that, you'll require me to take a dependency on the main repository that contains the common library. That repository is 39 separate projects in a single solution, is 1 gigabyte in size, and it won't build on my machine from the current repository. Why don't we just split out the common projects into their own repositories?

SHE: >_<