Git – shared CD (Continuous Deployment) for multiple Git repositories

continuous integrationcontinuous-deliverygitJenkinsrepository

Long story short

What is the best way to design a shared CD for multiple Git repositories (frontend and backend)?

I'm struggling to find the best design for our CD.

The whole picture (The details can be changed)

Frontend – Angular 2 (Javascript), no artifactory.
There will be only a single frontend in the project.
Backend – ASP.NET (C#), no artifactory.
There will be only a single backend in the project.
Git repository on Git TFS for each.
Server for Jenkins.
Multiple machines, Dgroup, for deploying the last version from branch development from both the repositories – a playground for each developer.

My goals:

development branch must be protected by pull request and build must pass. In more details: Jenkins need to confirm that the last version of the branch who activated the pull request os compiled and the tests of his repository are pass. Also the integration tests with the other repository won't fail(The tests of the frontend). Meaning for each pull request in every repository, Jenkins need to "unified" the build for both the repositories. If every test of each repository passes, then the pull request is ready to be approved. Else, the pull request is denied.
After each approved pull request, Dgroupmachines should be updated to have the latest version of the product (services + frontend files).

Flaws:

I can't find a way to achieve my first goal from multiple reasons:

The frontend can't push any new feature from the beginning until the backend will finish the same feature from his side, else some frontend's tests won't pass. I don't know how it will effect the workflow of the frontend and if it is the best practice.
Imagine a situation when the development branch in both the repositories is broken but no1 yet wrote the right test that will find the bugs (For now the build passes). Now the frontend create a pull request which contain a test that find a bug in the backend. So the build don't pass so automatically his pull request is denied.

Now the backend fix the problem and try to pull request. Now his change also fail a test of the frontend. So his pull request also denied.As a result no one can pull request anymore.

My CD design, which I did not describe due to the flaws, prevent me from going forward. What is the best practice to design a CD for multiple Git repositories?

Best Answer

IMHO the root cause of your problem is attempting to manage an entire system (frontend + backend) while still allowing each one of the parts to move independently and without the proper tool support for it.

A better (IMHO) approach would be to design your CI/CD approach with the system in mind. Any change to either part of the system (or a combination thereof) should always go through testing the entire system. This way you ensure the focus always remains on the entire system's integrity. I know, this goes against the micros-services trend (which I'm not very fond of, one of the reasons being exactly because they leave room for this kind of problems), but it is fundamentally what IMHO you need.

The complication is that the traditional CI tools, Jenkins included, aren't really designed for such approach. You could try to tweak it, but I imagine it's not an easy task.

Another problem is what I'd call absolute verifications vs just regression (or delta, if you want) verifications: for example you shouldn't reject a change to the frontend if verifications fails testing because the backend portion is not done - that testing didn't pass before, thus it's not a regression. Only after both the frontend and the backend functionalities were completed and the test managed to pass at least once a test failure should be blocking the commit - because it is now a regression. Donno if Jenkins supports such functionality.

You might want to take a look at ApartCI, it was designed with such system-oriented development in mind. Disclaimer: I designed it and I am the founder of the company offering it.

Hold on, skip merging

For this approach, you do not want to merge your feature_a into feature_b repeatedly.

Rebasing has been mentioned in other answers, but only for rebasing things onto master. What you want to do in your case is:

Start your feature_b from feature_a, i.e.:

git checkout feature_a
git checkout -b feature_b

Whenever feature_a changes while it is waiting to get merged into master, you rebase feature_b on it:
```
... commit something onto feature_a ...
git checkout feature_b
git rebase feature_a
```
Finally, as soon as feature_a has been merged into master, you simply get the new master and rebase feature_aonto it a last time:
```
git checkout master
git pull origin master
git checkout feature_b
git rebase --onto master feature_a feature_b
```
This final rebase will graft all commits that are dangling from the feature_acommit (which is now irrelevant as it has been merged into master) right onto master. Your feature_b is now a simple, standard branch going right from master.

EDIT: inspired from the comments, a little heads up: if you need to make some change which affects both features, then be sure to make it in feature_a (and then rebase as shown). Do not make it in two different commits in both branches, even if may be tempting; as feature_a is part of the history of feature_b, having the single change in two different commits will be semantically wrong and possibly lead to conflicts or "resurrections" of unwanted code, later.

Long story short

The whole picture (The details can be changed)

My goals:

Flaws:

Best Answer

Related Solutions

Continuous Integration / Deployment: Test on commit, pull request, or what

Git – Working on a branch with a dependence on another branch that is being reviewed

Hold on, skip merging

Related Topic