VictorOps is now Splunk On-Call! Learn More.

ChatOps and Using Hubot for Incident Response

Dan Holloran July 16, 2019

DevOps ChatOps Collaboration
ChatOps and Using Hubot for Incident Response Blog Banner Image

ChatOps is a method for using tools to execute commands, surface alert context and take action directly through chat. You can align human workflows with application and infrastructure health, making it easier to communicate and fix problems from a single tool. DevOps and IT teams are using chatbots like Hubot to execute commands directly from chat tools like Slack or Microsoft Teams. Now, with ChatOps, developers and sysadmins can collaborate with teammates in one tool, without needing to switch windows or applications to run scripts and commands.

Chatbots are essentially allowing teams to access the command line directly through their normal chat tool. And, with the rise of natural language processing (NLP) technology, more organizations are experimenting with more natural chat interactions and voice-recognition bots such as Amazon’s Alexa. With people at the helm of the services you build and maintain, ChatOps is a way to make systems work in a human-centric way – not the other way around.

So, we wanted to dive into one of our own favorite chatbots, Hubot, and take a look at how DevOps and IT teams are using Hubot for incident response.

An overview of Hubot

Hubot is one of the most well-known chatbots on the market today. It’s a Node.js application originally developed by Github for internal use only. But, eventually, Hubot was released to the larger public as an open-source service. The chatbot is extensible with CoffeeScript or JavaScript and offers numerous community scripts – but also allows you to build out your own custom scripts.

Hubot gives DevOps and IT teams an actionable way to automate tasks throughout the deployment lifecycle and incident management process. You can run tests through staging and production environments via in-chat commands or execute server restarts directly from chat applications. ChatOps is about taking action where people are already collaborating – and Hubot can serve as an integral part of this workflow.

Monitoring & Incident Response

How do you know when you need a chatbot?

For the most part, any DevOps or IT team can leverage chatbots in some capacity. But, the extent to which you use chatbots can vary based on your unique situation, your incident management maturity and the tooling you already have available. When assessing your need for a chatbot, you’ll need to look at the current functionality of third-party integrations and tools you’re currently using.

Chatbots can act as backup plans in case of third-party service outages or add flexibility to current workflows. You can establish ways to execute commands through Hubot that will function through multiple different channels (e.g. Slack, email, SMS, etc.). Or, you can simply use Hubot through one channel for one specific service in your system. ChatOps offers great amounts of flexibility and speed into human workflows. But, it also provides security and redundancy toward a team’s dependency on third-party applications and services.

The importance of ChatOps for incident response

ChatOps can help IT operations and software development teams facilitate a culture of DevOps. You can bring alerts into a single Slack or Microsoft Teams channel and centralize all of your communication and action. Not only can you tag a teammate and loop them into the conversation but you can easily run commands to run tests, execute scripts and surface runbooks to responders. It allows you to communicate in real-time and take action in a highly visible, collaborative way.

Developers and IT operations teams are already using a single source of truth for communication – so why should you move actual scripts, tasks and commands to a disparate location? ChatOps is moving interactive, automated incident response processes to the places where people are already looking for information and working. Improved transparency and collaboration leads to a tightened DevOps relationship between engineering and IT teams – speeding up incident response and remediation while also speeding up future software delivery.

Integrating people, processes and tools with ChatOps

One of the biggest problems faced by software developers and IT organizations today is cohesive transparency from idea to service implementation. From development to production, numerous things can go wrong – and teams don’t have a way to quickly find problems. Then, when they do find the root cause of an incident, there’s not an established plan for addressing the issue.

ChatOps and chatbot tools like Hubot facilitate a culture of DevOps – helping people, processes and technology collaborate more effectively. ChatOps is about more than tools, it’s about bridging the gap between humans and technology problems. On-call alerting integrations with Hubot and Slack can lead to centralized workflows where everything from incident detection to post-incident reviews can be conducted in one place. And, on-call responders can find the context they need in-line with the people they need to communicate with.

Learn more about ChatOps and how DevOps teams are applying it to the incident response and incident management process. Download our free O’Reilly eBook, ChatOps: Managing Operations in Group Chat to see exactly how you can implement ChatOps on your own team.

Let us help you make on-call suck less.

Get Started Now