Currently our Python application is deployed like this:
- Development team work on issues, commit code and create a Pull request
- Pull request is integrated to Development branch
- QA and Operations team test & deploy Development branch and then move it to Staging
- If all is good after a week we move it to Production.
Python code is stored in a Git repo and in production servers is just updated (git pull) based on latest Branch (Master), services restarted and all is good.
Today I came across a Google based build system (Bazel).
Our python app uses:
- Flask + Celery (external Python libraries: vnc, vmware, ssl, json, logging, math, etc.)
Externally:
- RabbitMQ
- PostgreSQL
- Ngnix
- Lighttpd
For testing we use Unittest.
What would be the benefits of deploying a Build system as part of our Dev pipeline?
Best Answer
Instead of thinking about the potential benefits of a potential tool, think about the actual issues and bottlenecks you encounter in your workflow—only then you should focus on the tools which actually solve those issues.
For instance, you may have a slow feedback cycle between developers and QA. Imagine a developer commits the changes on Monday at 5 p.m.; the working day ends at 6 p.m. The commit contains a regression. Would the developer learn about this regression a few minutes later, and be able to work on a fix when the code is still fresh in her head? Or maybe tomorrow morning, when she arrives at work and still remembers, although not very well, what she did yesterday? Or maybe only on Friday, when she has absolutely no idea what she was doing the day the regression appeared?
By running unit tests, build platforms make it possible to shorten the cycle between the time the regression is introduced and the time the developers are alerted about it. The sorter the better. If it's a few seconds after the “bad” code is introduced, great (this is the benefit of background compilation in some IDEs). If it's a matter of minutes, it can work. If it's a week, chances are you're wasting hours of painful debugging.
Or maybe you want to know the health of the project: an increase in failed builds usually indicates a problem. A build system usually makes it possible to visually identify such trends over days or months or the entire life of the project.
Or you may notice that developers spend time working on adding features, while there are blocking issues. They commit non-working code, their colleagues check out this code and are unable to progress or work effectively. Suddenly, most members of the team are found with code which won't compile, and all try to solve at the same time the same issue, leading to wasted hours and a mess.
A build platform would help here, by providing a visual of build failures. If a build fails, this should be an immediate response of a team: other members should avoid updating to the problematic revision; the author of the commit which caused the build to fail should focus on fixing the build (eventually soliciting help from pairs).
Maybe you simply want to give QA and Ops people a possibility to get back to an older revision either to test if a bug already existed or because the more recent revision introduces a blocking regression in production. In this case, a build system coupled with artifact repository will let them switch with ease between different revisions, without having to contact the developers.
Those are four cases of eventual issues which could be solved by a build platform. Look at the actual issues you encounter and:
Either search for the tool or a change in your workflow which solves the issue,
Or describe those issues on Programmers.SE, asking what should be done to solve them.
It might be that you actually need a build server, or a Continuous Integration platform. Or maybe you need to introduce DevOps in your organization. Or a simple tiny change in the way things are done is largely enough. Remember to focus on your issues, independently of the actual tool: