I'm going to take a quick first cut at this (great Q BTW!):
Would imposing a structure on the large project (i.e. into smaller
sub-projects) slow the compiler down?
Not by enough that it matters, the overhead is actually in Maven invocations.
Also, I have a slight concern on what impact this might have editing
time in IDEs (we principally use Intellij). Intellij seems to build
each project in turn through the dependency tree - i.e. if C depends
on B depends on A, and I change A, it won't try to build B unless A
compiles, and so on. Arguably that's advantageous, but I have found
that if - for example, I change an interface in A that is widely used
in B and C, it takes some time to fix all the errors from that
change...
Different IDEs have their different strengths with regards to Maven bindings and dependency management. The current state of play seems to be that it mostly just works on the latest Eclipse, Netbeans and IntelliJ - but you will have to teach your developers the emergency double whammy of "Refresh source files from disk and rebuild all related maven projects".
I find I'm having to do that less and less these days though. Having an SSD drive makes a massive difference here BTW.
snip factory classes paragraphs
Dependency management is incredibly important, regardless of what technology (Maven/Ivy/whatever) use use to help you implement it.
I'd start by getting the extensive reporting out of the Maven dependency plugin and take stock of what you've got. Generally speaking you set the dependency in the dependency management of the POM as high up the food chain as possible, but no higher. So if two of your submodules use an external dependency, then haul that into their parent POM and so on and so forth.
Upgrading external JARs should always be done as a proper mini-project. Evaluate why you're upgrading, alter source code to take advantage of any new features/bug fixes etc. Just bumping the version without this analysis and work will get you into trouble.
So, in general, my questions are:
Does anyone have any experience of breaking up large projects? Are there any >tips/tricks that you would be willing to share?
- Interfaces and Dependency injection are your friend.
- Michael Feather's book on dealing effectively with legacy code is a must read.
- Integration tests are your friend
- Splitting the sub projects into foo-api (interfaces only!) and foo-core and having modules only depend on the foo-api helps a great deal and enforces separation
- Jboss Modules and/or OSGi can help enforce clean separation
What impact did this have on your development and build times?
Very minor impact on dev and build times - a massive gain in time for our overall continuous delivery pipeline.
What advice could you offer on structuring such a break-up of such a project?
Do the little things right and the big things tend to fall out cleanly afterwards. So split things off bit by bit - don't do a massive restructure of the whole lot unless you've got a high percentage of coverage with your integration tests.
Write integration tests before the split - you should more or less get the same result(s) after the split.
Draw diagrams of the modularity you have now and where you want to get to. Design intermediate steps.
Don't give up - some Yak shaving now builds the foundation for being able to "rapidly build and refactor without fear" Shameless plug -> from Ben and I's The Well-Grounded Java Developer.
Best of luck!
Best Answer
You can test that a jar is sealed by defining a class to exist in a package used in the jar file. If the jar is sealed, all packages are normally sealed, else you can seal them on a per-package basis. The class you create should then try to access protected or default members of a class in that package.
For example, if the jar has a class
com.example.Widget
where the packagecom.example
is sealed andWidget
has protected/default members, create a classcom.example.Test
that attempts to access those members. This should fail.The reason for sealing follows from a description of how to test it: you want your code to be self-contained and users of the jar should only use public members. Packages are not required to be unique across jars (and bin folders in your IDE) used by a program. You are able to define a class in the same package as in a jar and access protected/default members that might cause problems. Often, default visibility is used as an analog to C++'s
friend
keyword by allowing other classes in the same package the ability to perform actions that are only safe because the same programmers work on both classes and understand their internals. Perhaps a default visibility method forgoes some bounds checking in the name of performance, with the understanding that client code will never access it. For example, theString
class in Java has a few of these types of methods since String handling has to be lightning-fast because a large amount of code relies on it.BigDecimal
is built on top ofBigInteger
and uses default access to optimize a few operations that other users of the class ought not to use.TL;DR: Classes in the same package might access non-public members of each other for performance or simplicity. This access might not check invariants and preconditions. One can put classes in the same package in multiple locations in the classpath. Sealing a package or jar restricts client code from accessing these methods, because it might break stuff otherwise.
This will not cause any problems for users of the jar. The classloading mechanism that accesses resources (e.g. class files) in jars fully supports sealed jars and packages.
Related reading: Sealing Packages within a JAR File