Build System – How to Cope with Compiling a Large Code Base

build-systemdesignproductivity

Although I can code, I don't yet have any experience with working on large projects. What I did so far was either coding small programs that get compiled in matter of seconds (various c/c++ exercises like algorithms, programming principles, ideas, paradigms, or just trying out api's…) or working on some smaller projects that were made in a scripting language(s) (python, php, js) where no compiling is needed.

The thing is, when coding in a scripting language, whenever I want to try if something works – I just run the script and see what happens. If things don't work, I can simply change the code and try it out again by running the script again and keep on doing that until I get the result that I wanted.. My point is that you don't have to wait for anything to compile and because of that it is quite easy to take a big code base, modify it, add something to it, or simply play with it – you can see the changes instantly.

As an example I will take WordPress. It is quite easy to try and figure it out how to create a plugin for it. First you start by creating a simple "Hello World" plugin, then you make a simple interface for admin panel to familiarize yourself with API, then you build it up and make something more complex, in the mean time changing how it looks a couple of times.. The idea of having to recompile something as big as WP over and over again, after each minor change to try "if it works" and "how it work/feels" just seems inefficient, slow and wrong.

Now, how could I do that with a project that is written in a compiled language? I would like to contribute to some open-source projects and this question keeps bugging me. The situation probably differs from project to project where some of them that were pre thought wisely will be "modular" in some way while others will just be one big blob that needs to be recompiled again and again.

I would like to know more about how this is done properly. What are some common practices, approaches and project designs (patterns?) to cope with this? How is this "modularity" called in programmers world and what should I google for to learn more about this? Is it often that projects grow out of their first thought proportions which becomes troublesome after a while? Is there any way to avoid long compiling of not-so-well designed projects? A way to somehow modularize them (maybe excluding non-vital parts of program while developing (any other ideas?))?

Thanks.

Best Answer

Just like it has been said, you never recompile the whole project each time you make a small change. Instead you only recompile the part of the code that has changed, as well as all code depending on it.

In C/C++, compiling is pretty straightforward. You compile translate each source file into machine code (we call them object files *.o) and then you link all your object files into one big executable.

Just like MainMa mentioned, some libraries are built into separate files, that will be linked dynamically at run-time with the executable. These libraries are called Shared Objects (*.so) in Unix and Dynamically Linked Libraries (DLL) in Windows. Dynamic libraries have many advantages, one of which being that you don't need to compile/link them, unless their source code effectively change.

There are build automation tools that help you:

  • Specify dependencies between different parts of your source tree.
  • Launch punctual, discreet compilations only in the part that was modified.

The most famous ones (make, ant, maven,...) can detect automatically which parts of the code has been changed since last compile, and exactly what object/binary needs to be updated.

However, this comes a the (relatively small) cost of having to write a "build script". It is a file containing all the information about your build, like defining the targets and their dependencies, defining which compiler you want and which options to use, defining your build environment, your library paths, ... You maybe heard about Makefiles (very common in the Unix world), or build.xml (very popular in the Java world). This is what they do.

Related Topic