Todd Vernon - November 26, 2014
DevOps is full of counterintuitive patterns. Agile software development has lead to faster product innovation and happier customers. But, system change has also led to more problems making it to production.
Continuous deployment has made it easier to correct problems and push to production. But, more pushes to production require more people to be aware of what is going out to help control customer expectations.
The fact is that for a lot of SaaS business, especially in e-commerce, Cyber Monday is the real test of their systems. In many cases, a seasonally-driven SaaS business reaps 60% or more of their revenue in Q4 of each year. So how do the Agile and DevOps patterns mesh with a business that operationally can vary wildly this time of year?
The answer we hear more often than not this time of year is: we are in lockdown. Simply put, this means that many companies establish a window that starts sometime in early November and lock down production during that time. While this can sound old-school to smaller nimble companies, it’s an extremely valid pattern for established companies.
In our forthcoming State of On-Call report, we found that most companies, regardless of size, have not done an analysis of what downtime costs them annually. However, they do know intuitively that even having a hiccup in production at this time of year can be disastrous.
So lock it down and go home? Not exactly.
This is actually a great time of year to look inward on your processes and data. While your machine is doing the work you set it up for all year, take this opportunity to look for ways to improve it in the coming year.
Review your retrospectives (port-mortems) and look for reoccurring patterns.
Look back on your incident reports in order to fine-tune monitoring and reduce noise in the system.
Inspect your on-call policies to see what is working and what is not.
Also, this is a perfect time of year to look at new technologies. At VictorOps, we will be reviewing some great tools from our partner network. A huge percent of our customers use New Relic and during the break, we are going to take a look and see how we can leverage this toolset to achieve even more.
Internal metrics reporting is on our plate as well. The old saying “The cobbler’s children have no shoes” often plagues companies. We don’t want this to be the case in our company and therefore, are really drilling into in-depth metrics of the VictorOps business. In coming months, you will see those play out in terms of new products that you can use to run DevOps in your business.
All that aside, we wish you the best of luck with Cyber Monday. We are here in the trenches with you and will do everything we can to ensure your systems (and revenue) remain up!