From startups to enterprise organizations, shifting to DevOps is a significant investment for any company. Even though DevOps has been around for about 10 years, we are still navigating the cultural and technical implications of DevOps practices.
We wanted to better understand these challenges and implications as companies invest in DevOps, so we sat down for a Q&A with Forrester analyst Chris Condo. Chris has a background in software engineering at Microsoft and focuses on understanding the application development and delivery market. He shared his learnings from researching how DevOps continues to transform the industry, including the role of technology, and the future of ITSM.
All software organizations should be executing a DevOps plan, regardless of their current level of maturity.
Companies should first determine core business goals such as: improve uptime, decrease cycle time, increase customer satisfaction, etc. These goals should be driven by “Why”, as in, “Why are we in business to begin with?” How do we improve customer satisfaction, increase retention, increase purchasing, reduce downtime, reduce time from idea to production, obtain customer feedback sooner, etc.? Teams that first focus on automation often miss the key reasons for adopting DevOps.
Next, learn about the current state of product development by performing a lean value stream mapping of each process to learn and determine bottlenecks ripe for automation and modernization. Organizations should note that bottlenecks in development are often cultural issues and should first use this exercise to optimize their organizations around product lines. Then they can think about where to apply automation, tools, and technology.
The growing DevOps trend could construe the ITSM struggles as irrelevant. However, it still represents a big chunk of IT management concern. How can modern IT, Ops, and Dev teams address management concerns while simultaneously supporting a transition to DevOps?
IT, Ops, and Dev teams address the transition to DevOps by collectively embracing automation, collaboration, and a software engineering mindset. It’s wrong to consider ITSM as irrelevant because it’s ITSM that’s operating and maintaining the very systems that support applications in production. Instead, we need to consider the ways that ITSM is evolving, and continues to evolve, to take on new challenges such as Cloud, Hybrid-Cloud, DevOps, IoT, AI and ML, plus a host of others. Each new technology brings new challenges to ITSM professionals.
For DevOps specifically, ITSM needs to provide dev teams with self-provisioning capabilities for all needed environments, from Dev to Production. Where possible, they need to share the burden of maintaining DevOps toolchains in a collaborative manner with developers; working with them to assemble, maintain, and constantly curate the best possible tools to keep the DevOps pipeline moving efficiently, allowing developers to focus more on delivering value to customers.
ITSM can further assist development by breaking down barriers to compliance and governance by using automated tools to configure, audit, and maintain systems of development, testing, and production. And lastly, ITSM and Dev need to build automated and connected systems of production monitoring that allow IT and Dev to collaboratively resolve production issues, with each recognizing that they play an important role in delivering value to their customers.
Today, many software developers are no longer just writing code — they are managing their code in production and owning more of the customer experience. While this is an amazing responsibility, it also creates new challenges. What actions have you seen companies take to help engineers handle this responsibility?
Engineering needs automation to retrieve, scrub, and facilitate diagnostic information. Ownership must persist to enable behavioral and institutional knowledge of the products in development and how they operate in production. Leadership culture must support reinvestment of time to continue learning, automating, and applying learnings from production to minimize recurring issues.
Future tools will leverage machine learning to detect patterns early rather than later in the production cycle and eliminate the old-school methods of waiting for enough customers to complain.
Technology has empowered us to connect with others, make decisions, and act in unprecedented ways. What communication trends are you seeing ITOps teams adopt as they move along their DevOps journey?
The use of real-time messaging systems like Slack, Hipchat, and Stride integrated with the software delivery toolchain are enabling virtual teams to quickly assemble, share knowledge, view data, and analyze check-ins (for example) to quickly execute root cause analysis or post-incident reviews. This integration is what’s new.
Collaboration was already there, but it was typically uniformed and required a lot of guesswork for external stakeholders to make sense of it all. Tools that leverage AI + ML will more quickly illuminate and pinpoint data to assist recovery. Connecting these tools with the different sources of inbound information is the best way to enable a culture of communication.
Running solutions at large scale creates lots of data, from standard logs to metrics. What opportunities does all this data provide for modern operations teams?
To make sense of this vast sea of disparate and voluminous data, engineering needs upfront data architecture to plan for correlation and diagnosis. From there, they will need assistance from bots, AI, etc. to discern patterns. That means skills will need to evolve from understanding how to read a fault in a log file to how to specify any anomalies that ML should identify as potential error conditions.
Data scientists could also become part of the shared resource team that assists product teams in developing maintainable systems.
Any thoughts in summation?
Each role needs to focus on its core capability.
IT Operations should focus on infrastructure, network, and intelligence related to the scale, performance, and resiliency of that infrastructure regarding those concerns. They adopt automation, self-healing infrastructure, redundancy, and own the overall uptime and connectivity of the IT fabric running the production environment.
During product development, IT Ops plays a critical role as best practices advisor for examining planned software architecture and providing guidance on how design decisions are impacted by real world concerns. They can also plan ahead, understand build out, or in the case of cloud, plan additional resources into the budget.
In the case of cloud, they are also experts in elastic systems, auto-scaling, and bring best practices for controlling consumption. As more work loads shift to cloud environments, IT Ops grows expertise in understanding Ops as a Service. Complex infrastructure arrangements such as hybrid and multi-cloud will continue to keep IT Ops busy into the future.
Product development teams are owned by Dev, product owners, and QA, owning the product development phase and product ownership phase. These teams need the capability to not only develop products, but must also own the automation of testing, continuous integration, and the ability to instantiate dev, test, and integration environments with guidance from IT Ops. Product health, maintenance, and diagnostics are owned by product teams. To enable this capacity, they should build/buy systems of automation.
From a DevOps perspective, leaders need to untangle the core requirements of each role so they don’t get conflated. Dev and Ops each need DevOps capability, but it’s focused on their area of responsibility. Devs need DevOps to help automate product development and ownership. Ops needs different DevOps skills and expertise to own and maintain infrastructure of production systems. The other way of looking at this is a production Ops person should not be responsible for a CI/CD system. Likewise, a developer should not own production ops.