Java – Separating java projects

javamavenproject-structure

I have a large java project, and we use maven for our build cycle. This one project is used extensively – in other projects, in various applications, some of which are contained in it and some which are elsewhere… To be honest, it's a bit of a mess (various bits added on at different times for specific purposes), and I'd like to clean it up a bit. Also, it's not fully tested (lots of bits have been added on without proper unit and integration testing), and there are some tests that take a long time to run or don't actually pass… (uh-oh) – so tests are switched off in the maven build cycle (again, uh-oh).

I am thinking about separating this large project into smaller specific projects, such that the 'final' sub-project (or several sub-projects) would pick up the various sub-projects that it would need.

My thinking is as follows:

if I separate the big project into various sub-projects, this makes it clear what each project's responsibility is.
by separating into sub-projects, I can then clean up the testing of each sub-project individually, and turn on the testing for that sub-project in the maven build cycle.

I am slightly concerned about what impact this might have on the build time.

Would imposing a structure on the large project (i.e. into smaller sub-projects) slow the compiler down?

Also, I have a slight concern on what impact this might have editing time in IDEs (we principally use Intellij). Intellij seems to build each project in turn through the dependency tree – i.e. if C depends on B depends on A, and I change A, it won't try to build B unless A compiles, and so on. Arguably that's advantageous, but I have found that if – for example, I change an interface in A that is widely used in B and C, it takes some time to fix all the errors from that change…

Another question is how to use factory classes. Some aspects of the project depend on external jars. Occasionally (thankfully not often) these are updated, and we have to migrate. We tend to handle this by using a Factory class which points to the correct versions of the external code (so we don't have to change all the implementations throughout the code base).

At the moment this is all in the large project, but I am thinking that by switching to sub-
projects, I could develop a new project for the implementation of new external code, make sure that sub-project is fully functional and tested, and then switch the dependencies / factory class in the user project. However, this is made more complicated by the extensive use of interfaces throughout the large project. For example

sub-project A – contains interfaces
sub-project B – depends on A for interfaces and old external jar
sub-project C – depends on B (and thus A and old external jar), and contains a Factory class that uses B's implementations of interfaces

If I need to change the external jar of B, I can:

create sub-project B_ii – again depends on A, and now the new external jar
once fully functional, I can add C's dependency to B_ii, and change the Factory class to use the new implementations of interfaces.
when that's all working, I can then remove C's dependency to original B, and if desired, remove sub-project B.
Is that a sensible way of going about this?

So, in general, my questions are:

Does anyone have any experience of breaking up large projects? Are there any tips/tricks that you would be willing to share?
What impact did this have on your development and build times?
What advice could you offer on structuring such a break-up of such a project?

Best Answer

I'm going to take a quick first cut at this (great Q BTW!):

Would imposing a structure on the large project (i.e. into smaller sub-projects) slow the compiler down?

Not by enough that it matters, the overhead is actually in Maven invocations.

Also, I have a slight concern on what impact this might have editing time in IDEs (we principally use Intellij). Intellij seems to build each project in turn through the dependency tree - i.e. if C depends on B depends on A, and I change A, it won't try to build B unless A compiles, and so on. Arguably that's advantageous, but I have found that if - for example, I change an interface in A that is widely used in B and C, it takes some time to fix all the errors from that change...

Different IDEs have their different strengths with regards to Maven bindings and dependency management. The current state of play seems to be that it mostly just works on the latest Eclipse, Netbeans and IntelliJ - but you will have to teach your developers the emergency double whammy of "Refresh source files from disk and rebuild all related maven projects".

I find I'm having to do that less and less these days though. Having an SSD drive makes a massive difference here BTW.

snip factory classes paragraphs

Dependency management is incredibly important, regardless of what technology (Maven/Ivy/whatever) use use to help you implement it.

I'd start by getting the extensive reporting out of the Maven dependency plugin and take stock of what you've got. Generally speaking you set the dependency in the dependency management of the POM as high up the food chain as possible, but no higher. So if two of your submodules use an external dependency, then haul that into their parent POM and so on and so forth.

Upgrading external JARs should always be done as a proper mini-project. Evaluate why you're upgrading, alter source code to take advantage of any new features/bug fixes etc. Just bumping the version without this analysis and work will get you into trouble.

So, in general, my questions are:

Does anyone have any experience of breaking up large projects? Are there any >tips/tricks that you would be willing to share?

Interfaces and Dependency injection are your friend.
Michael Feather's book on dealing effectively with legacy code is a must read.
Integration tests are your friend
Splitting the sub projects into foo-api (interfaces only!) and foo-core and having modules only depend on the foo-api helps a great deal and enforces separation
Jboss Modules and/or OSGi can help enforce clean separation

What impact did this have on your development and build times?

Very minor impact on dev and build times - a massive gain in time for our overall continuous delivery pipeline.

What advice could you offer on structuring such a break-up of such a project?

Do the little things right and the big things tend to fall out cleanly afterwards. So split things off bit by bit - don't do a massive restructure of the whole lot unless you've got a high percentage of coverage with your integration tests.

Write integration tests before the split - you should more or less get the same result(s) after the split.

Draw diagrams of the modularity you have now and where you want to get to. Design intermediate steps.

Don't give up - some Yak shaving now builds the foundation for being able to "rapidly build and refactor without fear" Shameless plug -> from Ben and I's The Well-Grounded Java Developer.

Best of luck!

Related Solutions

Spring MVC – Creating Web Services: One Project or Multiple Projects?

It depends on how you want these web services to scale. For example, if your 4 GET calls are called 1 million more times per day than the 1 PUT request, then you'll want to split them out so you can load balance them more easily.

Java – Best Way to Structure a Git Repository for Maven

You have 2 options:

by git way: use submodules. Here is a documentation how git manage submodules [git submodules][1]. I personally didn't use it but it looks to fit your problem.
by maven way: in maven it is not mandatory that your root project (configuration) to be hierarchically the parent directory of all your projects. You can have a structure like that:

configuration
 +-- pom.xml (configuration:XXX)
project1
 +-- pom.xml (project1:1.0-SNAPSHOT)
 !
 +-- module11 
 !     +-- pom.xml (1.0-SNAPSHOT)
 +-- module12
       +-- pom.xml (1.0-SNAPSHOT)
project2
 +-- pom.xml (project2:2.0-SNAPSHOT)
 !
 +-- module21 
 !     +-- pom.xml (2.0-SNAPSHOT)
 +-- module22
       +-- pom.xml (2.0-SNAPSHOT)

configuration, project1 and project2 are on the same directory level and each one could be a git repository. When build project1 or project2 you run the maven command from project1 or project2 level and maven will try to fetch the parent (configuration) from maven repository not from parent directory. You should pay attention to versions. I would recommend in project1 or project2 to keep a reference to a parent (configuration) with a release version.

To make a release you have to do it in 2 steps: release configuration first and release project second. Project1 and project2 can evolve independently and doesn't have to have the same configuration version as a parent.

Just for special cases when you want to have both configuration and projects as SNAPSHOT versions, in project1 or project2 you can use the <relativePath> tag inside <parent> tag to point to your local path of configuration. I don't recommend this because will create problems on development environment (at least for me in Eclipse)

I apologize for my English. [1]: http://git-scm.com/book/en/Git-Tools-Submodules

Best Answer

Related Solutions

Spring MVC – Creating Web Services: One Project or Multiple Projects?

Java – Best Way to Structure a Git Repository for Maven

Related Topic