DevOps

Jenkins and Advanced CI/CD

Amazon deploys every 11.7 seconds. Netflix ships changes without a single minute of downtime. Both combine Jenkins-style pipelines, Canary releases, and feature flags - and never hit 'STOP SERVER' to deploy. This lesson covers exactly how that machinery works.

**Netflix** uses Canary releases for every recommendation algorithm change - monitors retention metrics on 1% of audience before full rollout
**Facebook** launched Instant Articles via feature flags - enabling it incrementally for publishers without touching the main product
**Jenkins** is used at Toyota, NASA, and major banks - where self-managed CI/CD infrastructure is a compliance requirement

Jenkins Pipeline: Declarative vs Scripted

Jenkins is the oldest CI server, released in 2011. Despite its age, it remains the enterprise standard: thousands of plugins, deep flexibility, and on-premise control. The backbone of modern Jenkins is **Pipeline as Code**: a Groovy-based `Jenkinsfile` checked into the repository alongside application code.

Two syntaxes exist: **Declarative** - structured, readable, covers 90% of use cases; **Scripted** - full Groovy, maximum flexibility, harder to read. Starting with Declarative and moving to Scripted only when necessary is the recommended approach.

**Shared Libraries** are Jenkins' mechanism for reusing Groovy code across Jenkinsfiles in different projects. Libraries live in a separate Git repository and are loaded via `@Library('my-shared-lib')`. Common logic - Docker builds, deployments, notifications - lives in one place instead of being duplicated.

What is the primary advantage of storing the Jenkinsfile in the project repository?

Blue-Green Deployment

Traditional deployment: stop the old version, start the new one. The service is unavailable during that window - downtime. **Blue-Green deployment** eliminates this: two identical production environments are maintained (Blue = current version, Green = new version), and traffic is switched instantly via the load balancer.

Cost: Blue-Green requires double the infrastructure. In cloud environments this is resolved through autoscaling - the Green environment spins up for the deploy and is destroyed ~30 minutes later after stability is confirmed. Kubernetes implements Blue-Green by changing the Service selector between two Deployments.

What happens to the old Blue environment after traffic has been successfully switched to Green?

Canary Releases

The name comes from the mining practice of carrying a canary into a shaft to detect toxic gas. If the canary died, miners escaped. **Canary releases** apply the same principle: a new version is routed to 1-5% of users first. If metrics hold steady, the percentage increases gradually. Problems surface on a small audience, not on everyone.

Canary in Kubernetes is implemented with two Deployments sharing a Service, with replica counts adjusted to match the desired traffic split. The more advanced approach uses Istio or Argo Rollouts, where traffic weights are independent of replica counts. Argo Rollouts supports automated promotion or rollback based on Prometheus metrics.

**Canary vs Blue-Green:** Blue-Green switches traffic all at once (0% to 100%). Canary moves traffic gradually (1% to 10% to 100%) with observation periods. Canary suits changes with uncertain performance or UX impact. Blue-Green suits fast deploys where quality is already validated.

How does Canary deployment differ from Blue-Green?

Feature Flags

Canary splits users randomly: 1% gets the new version. Feature flags give finer control: a specific user, organisation, country, or A/B group sees a new feature while others do not. The code is already deployed to 100% of servers - it is simply gated behind a runtime check.

The pattern **decouples deploy from release**: code ships frequently (potentially multiple times per day), while the feature is enabled for users as a separate decision. This reduces risk and supports trunk-based development without long-lived feature branches.

**Flag cleanup:** feature flags are temporary. A flag enabled at 100% should be removed from the codebase within one to two weeks. Accumulating dead flags is a well-documented source of technical debt. LaunchDarkly, Unleash, and Flagsmith are popular platforms with UI and analytics for flag lifecycle management.

Feature flags are the same as Canary releases, just with a different name

Feature flags control feature visibility for specific users. Canary controls the deployment version for a random percentage of traffic.

Both can be combined: deploy new code via Canary to 10% of servers, then inside that cohort enable the feature only for beta users via a feature flag.

A team wants to show a new interface only to enterprise accounts, without a separate deployment. What should be used?