VictorOps Case Study: Creating a Salesforce Integration with our Alert Ingestion API


Wait. A case study about our own company? Is this a perfect example of us drinking our own Kool-aid?


But, in our defense, it’s also a perfect example of how we used our own API to create an integration with our service that we needed as an internal tool. So how did a non-programmer go about making this happen? Was it magic?


This all came about because someone wasn’t afraid to ask questions. And that someone is Nate. Nate is on the ops team here at VictorOps and wanted a better way to interact with Salesforce, making it a more useful weapon in the support team’s arsenal. After attending DreamForce last month in San Francisco, he came back to the office armed with Salesforce knowledge and ready to dive into it.

Read on for more of Nate’s story…


What was the problem that you wanted to solve?

During my extensive career (almost a year and a half!!!) in operational support, I have noticed a gap in internal communication between the DevOps/IT teams who monitor infrastructure and the support teams that deal with the customers. Far too many times I have been troubleshooting an issue affecting a large pool of customers, sometimes an issue spanning over many days, only to find out later that it was an issue with a down server, known bug or something of that sort.

It is illogical for the DevOps/engineering team to waste time informing support of exactly what is going on as they should be working towards fixing the issue as soon as it is discovered. The ideal option would be giving the support team access into the DevOps world so they can follow along during any type of crash/down time thus, allowing them to correlate issues they are seeing with support to those with the infrastructure without having to bother those working to resolve the issue.

Simply giving the support team a log in to view the timeline would give them this level of insight into the DevOps world, but to be honest, this does not get them engaged nor does it encourage anyone to make correlations between infrastructure and customer facing issues.

Screen Shot 2013-12-10 at 10.41.31 AM

What was the solution you created?

Using the VictorOps Alert Ingestion API, I created an integration that sends customer support alerts coming in from Salesforce directly into the VO Incident timeline. Basically, when a customer sends an email to VictorOps support, there is a case created in Salesforce. Once the case opens in SF, an alert is sent to VO and from there, you can ACK the alert. When the case is marked as resolved in Salesforce, it shows up as resolved in our timeline as well.

You can see how adding the customer support inquiries directly into the VictorOps timeline comes into play in the above screenshot. Having the infrastructure and customer issues in the same place all in real time is a game changer for both teams. It allows for both teams to learn about the issues the others are dealing with, and it puts it in a way they can both understand.

An example: a server goes down and all parties are alerted. The engineers get to work and the support team gets ready for any associated support tickets. At first, this server is a complete foreign entity to operations and although the engineers know what it is meant to do, they may not be aware of the specific impact it will have on the customer. Over time both the operations and DevOps teams will learn about the servers and what they effect, allowing them to then set up alerts based on the level of importance.

“_Show me someone who presents sharing information between teams regarding how the company operates as a bad thing, and I’ll show you an idiot_.”

Have you ever created an integration like this before? If not, what were your major pain points / challenges?

I would never think to attempt to create and integration like this since my experience with programming is null. But after a little digging and some handholding from our VO programmers, it turned out to be fairly simple. The biggest challenge for me was creating something like this from scratch because I simply had no idea where to start. Now that it is built and I have picked up some programming knowledge, I am able to customize just about every bit of information I want going to the timeline, regardless of where it is created (email, web-to-case, manual case creation etc..).

Screen Shot 2013-12-10 at 10.45.22 AM

How was it working with the VictorOps Alert Ingestion API?

Creating the apex class and trigger was the hard part but other than that, using our Alert Ingestion API is a breeze. Now that I have the code built for Salesforce I can switch up the organization I point alerts to and adjust what type of alerts they are in a matter of seconds…all while never running into a single issue.

How has the Salesforce integration helped you? It has given me a better understanding of how both Salesforce and VictorOps work individually and more importantly, I now know how they can work together better.

Any advice to others looking to create their own integrations?

Map out exactly what you want to accomplish before you get started. Making changes along the way was not too complicated but it is definitely a benefit to define what you want ahead of time.

What were the big lessons you learned from doing this?

Basically everything is possible with this integration because any bit of information can be passed through. I underestimated this when I first began and set my sights too low. If I could go back and try again I would have gotten to the integration I have now in less time.

Ready to get started?

Let us help you make on-call suck less.