Joey Smith manages all server maintenance and monitoring for Axis 41, a company of integrated creative and technical teams that support major brands around the world. When he decided to stop his personal on-call insanity and at the same time strengthen his dev team, he turned to VictorOps. Here’s what he said…
“I didn’t go into a movie theater for a year and a half.”
I used to be on-call all the time. I didn’t take my wife out on dates. I didn’t get to spend time with my family. I found VictorOps and thought, “This will give me my life back.”
Now I have some sanity and control over my own life. And there is more responsibility and more ownership across our whole team. VictorOps is a big part of helping our team feel ownership for the commitment that they make.
On-call rotations that avoid things slipping through the cracks
Before we had VictorOps, if someone had the day off and needed to switch, the two people just managed it between themselves. The problem with that is sometimes Tyler thought Brett was on-call… and Brett thought Tyler was on-call. That’s when things slipped through the cracks.
Now we have a schedule that says exactly when Brett is on and when Tyler is on. We’ve put VictorOps up on a big screen in our work area, so now anyone in the company can walk up and see the bubble to determine who is on call.
We put it to the test when Amazon did a rolling outage of every server over the course of five days. Right in the middle of those five days, one of the guys on the team had a family emergency. Another one of our guys logged into VictorOps and took on call for him. It all happened so seamlessly. I wasn’t even aware of it until after the fact.
A web app designed in this decade
I looked at another on-call product before I found out about VictorOps. The other guys felt like a 1990’s web app – it didn’t have all of the features that I expect from a mature web app. VictorOps was designed in this decade. I could tell that VictorOps innovates, which means they can help us innovate too.
Before we implemented VictorOps, we used email integration quite heavily. It was easy to drop the email alerts right in and have VictorOps be the repository where all the events happen. We also integrate with Zabbix and soon Datadog.
A new definition of devs
When I was hired 2 ½ years ago, each developer was responsible for his or her own projects. None of them focused on administration, troubleshooting on-call issues, or other holistic knowledge you’d want in a development team today.
We’re building this team from scratch, and I think we’re defining what it looks like to have a responsible, cross-functional dev team. VictorOps helped get sanity back in my life and also identifies the areas where we need to focus. VictorOps gives me insight into what I should be doing to build and mature the team.
VictorOps is a comfort to me – now and as we mature our team.