People, Process, and Technology of Continuous Delivery
What is Continuous Delivery
Before we start to understand Continuous Delivery, we shall take a small detour of what DevOps is. It is a combination of cultural philosophies, practices, and tools that increase an organisation’s ability to deliver applications and services at high velocity. This, in turn, helps products and applications develop at a much faster pace as compared to organizations using traditional software development infrastructure and management processes. If you were born around in the 90’s you should have had a taste of software update frequencies that have changed recently.
Under DevOps, we have the philosophy of continuous delivery, which is a software deployment practice, where code changes are automatically built, tested, and prepared for release into production. When continuous delivery is properly implemented, developers will always have a deployment-ready build artifact that has already passed through a standardized test process.
Continuous delivery lets project teams automate testing beyond unit tests so that they can verify application updates across multiple dimensions before deploying to production environment. These tests performed by developers may include UI testing, load testing, integration testing, API reliability testing, and so on. This helps developers thoroughly validate updates and pre-emptively discover issues. With the availability of cloud, it is easy and cost-effective to automate the creation and replication of multiple environments for testing, which was previously difficult to do on-premises.
Why was there a need for Continuous Delivery?
Before developers could theorize DevOps, coders followed a waterfall model of development. It was a simple, less iterative, and one-directional concept of development.
So why was there is a need to change this model? The initial assessment and delivery plan in a waterfall model looks like a straight path to success as depicted in the picture below:
But things are not so simple as we had imagined it would be.
Some common pitfalls seen during a waterfall lifecycle is that errors can only be fixed during the code writing phase. When the client needs to provide feedback or request changes, it becomes nearly impossible to account for it in the current cycle of development. The testing phase is quite late in the developmental process and documentation takes up a lot of time of developers and testers.
Why Enterprises must enable Continuous Delivery?
IT organizations are facing a serious challenge. On one hand, they must enable business es to respond ever faster to changing market conditions, serving customers who use an increasing variety of devices. On the other hand, they are saddled with ever more complex systems that need to be integrated and maintained with a high degree of reliability and availability.
When a project is successfully launched it is at the time the user first interacts with the system and at that point the project team disbands. The system is now thrown over the
wall to operations, and making further changes involves either creating a new project or work by a “business as usual” team.
This creates several problems:
- Many developers have never had to run the systems they have built, and thus they do not understand the trade-offs involved in creating systems that are reliable, scalable, high performance, and high quality. Operations teams sometimes overcompensate for potential performance or availability problems by buying expensive kits that are ultimately never used.
- Operations teams are measured according to the stability of the systems they manage. Their rational response is to restrict deployment to production as much as possible, so they do not have to suffer the instability that releases of poor-quality software inevitably generate. Thus, a vicious cycle is created, and an unhealthy resentment between project teams and operations teams is perpetuated.
- Because there are several disincentives for teams to release systems from early on in their lifecycle, solutions usually do not get near a production environment until close to release time. Operation teams tend to have a tight control over the build and release of physical servers. This slows down the ability to test the functionality and deployment of solutions. It can be inferred that that the production readiness cannot be not assessed until it is too late to change architectural decisions that affect stability and performance.
- The business receives little real feedback on what is being built and if it is of valuee until the first release, which is usually many months after project approval. Several studies over the years have shown that the biggest source of waste in software development is features that are developed but are never or rarely used- a problem that is exacerbated by long release cycles.
- The funding model for projects versus operating expenses creates challenges in measuring the cost of any given system over its lifecycle. Thus, it is nearly impossible to measure the value provided to the business on a per-service basis.
- Due to the complexity of current systems, it is difficult to determine what should be decommissioned when a new system is up and running. The tendency is to let the old system run, creating additional costs and complexity that in turn drive up IT operating costs
The upshot is that IT operations must maintain an ever-increasing variety of heterogeneous systems, while adding new projects. In most organizations, IT operations consume by far most of the IT budget. If an Organization can drive operating costs down by preventing and removing bloat within systems created by projects, the organization would have more resources to focus on problem-solving and continuous improvement of IT services.
Criteria for successful Continuous Delivery
To enable Continuous Delivery to clients, the synergy of people, processes, and technology is of utmost importance. Each one adds value to the other and they are connected to allow for frequent releases with the same level of quality
Elon musk during one of the meetings had asked one of his employees to skip a meeting because he did not have anything to add to the conversation. In short, this is DevOps in practice at a meeting conference. Before setting up a team for a software development project stakeholder would need to be identified, ensuring that everybody delivering value to the business are working tightly together on the common goal of adding value to the project. Having highly motivated people with good collaboration is also necessary.
Putting top engineers with top-notch development skills together does not always produce a successful team. People collaborating across traditional boundaries and silos are at the centre of this journey.
With a highly motivated team there is a need for smooth mature business processes otherwise if processes bottleneck, they can be a hindrance to innovation. Take an example of an approval process that takes ages before the change is implemented. To acquaint an organization to Continuous Delivery the process of designing, building, testing and deployment of software should be well presented to each team member. Business processes should be such that they enable flow and collaboration.
There is another aspect of the process that does not get to the limelight. It is the process of clean code. Code refactoring (the process of restructuring computer code) is important for optimal performance, as is continually scanning for security holes. This can be done through code reviews and code analysis. Use technology to augment the process of clean up through dashboards and analytics. This can help in real-time decision making and help teams deliver faster code accurately.
Everybody wishes that continuous delivery could be an off the shelf package that could be bought from a supplier. Continuous Delivery is a methodology, like the Bhagwat Gita quote that Wisdom can neither be bought not be sold. Continuous Delivery is a complex mechanism of development lifecycle processes and there is no “all in one” package or tool available to bridge the gap. A developer will look for something that will provide quick feedback on the codes he or she has just written. A test engineer on the other hand concentrates on the integration of various development codes into a packaged product. Production engineers look for service degradations along the end to end customer journey experience. Due to the different needs, there are products, tools, and services that can help enable Continuous Delivery practices across different teams which can be used to make things easier.
What do Organisations Realise with Continuous Delivery?
Organisations need to culturally shift towards different thinking to be able to implement Continuous Delivery. It will hurt the organisation initially just like hitting the gym for the first time. The more often you engage in the new set of activities and processes, people will adapt just like the muscles in a body. It is especially important to note that a gym trainer will ask you to focus on the core muscles first. Similarly, the organisation must prioritise the processes that have the greatest impact.
Increasing revenue requires releasing the code/ product /service to the market faster. To achieve a faster Go to Market there must be even faster and reliable build and deployment frameworks. Dividing this work among traditional functional teams may lead to inadequate system design decisions that are discovered too late. Merging smaller teams into larger teams focused on common goals will minimize handoffs from team to team. By working together, early on, teams can foresee problems and design a well-thought-through deployment process. The ability to deploy quickly and correctly begins with involving from the very beginning those responsible for deployment.
Driving Down Costs
Rework is lost revenue. Driving down costs by reducing rework requires building quality into the process. With a Continuous Delivery approach when teams are not building with nightly intensity, they can fine-tune scripts, solve problems, and cross-train team members. The staff will have the capacity to work on improvements instead of fixing regular bugs. This will allow for a leap in the quality of code because team members are not involved in chaotic middle night ordeals to complete testing.
Retention of Good People
Good top-notch engineers quit because of systemic frustrations that paralyze the team’s effectiveness and decrease team morale. Even with a high Pay engineer would still quit if the job is not a motivator. A team using a Continuous Devilry approach allows for continuous improvement of critical skills that result in increased expertise. Expertise is not built in a day. It happens in an incremental and evolutionary fashion.
Technology for Continuous Delivery
Every modern business is technology driven. As it already has been stated that technology is a one the major criteria for Continuous Delivery and there is no all in one package that does all of it.
Build tools are programs that automate the creation of executable applications from source code (e.g., .apk for an Android app). The Building incorporates compiling, linking, and packaging the code into a usable or executable form. Build automation is the act of scripting or automating a wide variety of tasks that software developers do in their day-to-day activities.
Open Source Solutions: Rake, Ant
Continuous integration help automate the integration of code changes from multiple contributors into a single software project. The CI process is comprised of automatic tools that assert the new code’s correctness before integration. A source code version control system is the crux of the CI process.
Open Source Solutions: Jenkins
Version Control Tools
Version control systems are a category of software tools that help a software team manage changes to source code over time. Version control software keeps track of every modification to the code in a special kind of database. If a mistake is made, developers can turn back the clock and compare earlier versions of the code to help fix the mistake while minimizing disruption to all team members.
Open Source Solutions: git, SVN/Subversion
Testing tools are products that support multiple test activities right from build to test analysis. This helps achieve quality code for a program and find bugs.
Open Source Solutions: cucumber, selenium
Deployment Automation Tools
Deployment automation tools enable the software to be shifted to testing and production environments with the push of a button. Automation is essential to reduce the risk of production deployments. It is also essential for providing fast feedback on the quality of your software by allowing teams to do comprehensive testing as soon as possible after changes.
Open Source Solutions: Fabric, Spinnaker
Provisioning Tools help setting up computers or virtual hosts by installing needed libraries or services on them. The thing to remember is that ‘deployment’ does not, as a rule, include ‘provisioning’. Ansible or Puppet for server provisioning. This can also be done automatically by a service like Engine Yard, Heroku
Open Source Solutions: Ansible, Puppet
Artifact Management Tools
Artifacts management tools, also known as artifact repositories, are used to store, organize, and distribute artifacts (that is, binary files plus their metadata) in a single centralized location. This reduces the amount of time spent downloading dependencies from a public place.
An example of a Continuous delivery model.