Six weeks ago, we started using Kanban to manage our product deployments. Although our primary motivation was to improve deployment lead time (defined as the time from when code was “done” until it was released on the production site), the results have been more far reaching. In fact, given how little we actually changed about the way work gets done, the outcomes are downright surprising.
For the past 18 months, we’ve been pushing releases to production twice a day. Although this sounds good on paper, our lead times were pretty bad. Once a team was ready, they would often wait for 5 business days before seeing their code go live.
A source of waste
Under our old deployment regime, we managed our releases via a release calendar. Each day had an AM and PM slot. In practice, teams would predict when they’d be done and snag a slot. If a team wasn’t ready when their release slot came around, nobody released during that time frame. Historical analysis revealed that roughly 20% of our release capacity was wasted by teams walking away from slots.
Two months ago, I started reading David Anderson’s Kanban: Successful Evolutionary Change for Your Technology Business. Tired of hearing me wax eloquent about YANI (yet another new idea), Yan Philippe, one of our group managers, suggested we apply it to our code deployments. On Oct 10th, we retired the old release calendar and replaced it with a very simple Kanban board.
Kanban board for code deployments at msnbc.com
Here are some of the improvements we've experienced thus far. In another post, I'll pontificate as to why we saw some of these improvements:
- Lead time – Average lead time decreased from 5 business days to 1.5 business days
- Release slot utilization – increased from 80% to 100%; basically, a release slot is never wasted anymore
- Readiness – after a few weeks of adjustment, teams are completely buttoned up and ready to go as they put the sticky on the board. If things aren’t ready when it’s their turn, they simply lose their slot and the next team immediately releases.
- Collaboration – Although the release queue is mostly FIFO, teams can switch order based on business priorities. The best part about it is that the release engineer doesn’t have to get in the middle of the conversation. The discussion happens between business stakeholders.
- Releases per day – Sometimes, the release team squeezes in 3 releases per day. This never happened under the old regime. This is possible because anybody in the queue is ready to go on a moment’s notice. Also, the release team feels accountable for their lead time metric and works pretty aggressively to keep the “ready to release” queue small
- Continuous improvement – The release team is more in tune with the process bottlenecks and takes small steps to improve flow.