“Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.”
-- Martin Fowler, http://martinfowler.com/articles/continuousIntegration.html
In software development there is a spectrum of the degree to which a project team can automate integration of components, bug fixes, enhancements, and new features into a complete software product. In a 2000 article (revised in 2006), Martin Fowler describes the then-current state of this automation using the term Continuous Integration. Fowler describes the tools and processes used to achieve this automation and the benefits of achieving Continuous Integration. Since then, many more tools and processes have entered the industry and it is becoming harder to fit all automated software development projects into a single definition of Continuous Integration. To help facilitate analysis and conversation around software development automation, I want to offer a set of terms describing various features of automation, how those features can be combined to achieve different levels of automation, and the benefits derived from each.
Features of Automation
Reproducible Builds
- Builds are conducted from a version-controlled source of truth using a scripted process.
- Provides traceability and accountability for every build of the system.
- Previous builds can be reproduced with precision by another developer at a later point in time.
- Enables troubleshooting of production issues outside of the production environment.
- Eliminates the risk associated with variability of a manual build.
Automated Builds
- Builds happen without manual intervention, reducing cost and freeing resources.
- Build-time failures are identified quickly, reducing the cost of remediation.
Automated Unit Testing
- Fails the build if individual components do not pass tests of basic functions.
- Logical and functional failures in components are identified quickly, reducing the cost of remediation.
Automated Qualitative Analysis
- Every successful build is analyzed for standards compliance, common coding mistakes, and security vulnerabilities.
- Quality and technical debt can be tracked and managed to reduce the ongoing cost of maintenance.
Automated System Testing
- Comprehensive testing that fails the build if components are not integrating properly when running in a simulated production environment.
- Identifies runtime functional errors quickly, reducing the cost of remediation.
Automated Delivery
- Successful builds are automatically deployed to selected environments for manual exploratory and user acceptance testing.
 |
| Automation features at various levels |
Levels of Automations
Manual
- No automation.
- Builds are created by IDE export features, manually from the command line, or through other processes that are subject to human error and environmental variability.
- “It builds on my machine” is a common refrain.
- It is impossible to precisely reproduce a previous build with any reliability.
- Automated tests (e.g. JUnit, Selenium) may exist, but they are not run consistently and automatically.
- Individual developers may apply quality analysis tools, although inconsistently and with no formal enforcement mechanism.
Scripted
- Builds are automated using a scriptable/configurable build tool: Ant, Buck, Gradle, Make, Maven, etc.
- Build script may include running automated tests and qualitative analysis, but still lacks systematic formal enforcement.
Automated Integration
- Builds are triggered automatically (typically nightly).
- Automated tests (if present) are run.
- Qualitative analysis may be performed.
- Enforcement of testing and quality standards is now possible.
Continuous Integration
- Developers integrate their changes into a central repository daily.
- Builds are triggered continuously as changes come in.
- Automated testing and analysis provide immediate feedback if breaking changes are introduced.
- May include some level of automated system testing.
- If deployment has been automated, CI should include automatically delivery of each build to a shared testing environment.
Continuous Delivery
- A combination of rigorous standards compliance, up-front peer review, and automated testing at the unit and system level is sufficient that the team is comfortable promoting changes as soon as the lights turn green.
- Builds and deployments are fully automated.
- Any build that passes a final round of user acceptance testing can be promoted to production with a single click.
