Attendees of Tech Stack Forum 2016
Last week I had the pleasure of speaking on a panel at Sapphire Ventures Next-Gen Tech Stack Forum in San Francisco.
Obviously, I was excited to join the discussion, but as a participant the event crystallized not only where the larger software development market is relative to microservices, container technologies (like Docker), continuous integration and deployment; but also provided insight into where DevOps best practice is heading in the coming years.
Spoiler Alert: Microservices are where it’s all going, Container systems like Docker or **Kubernetes are a huge enabler of that vision, code ownership flourishes in this environment, and DevOps promotes this culture and ecosystem of tools that unlock the huge social gains of all of it.**
Let’s unpack that long sentence a bit.
Microservices are not a new concept, they existed a decade ago by another name, Service Oriented Architecture (SOA). The idea of SOA, and now Microservices, was to break up huge monolithic software architectures into more easily adaptable parts that are loosely coupled and easier to implement. It was good idea then, and is still a good idea today. However, SOA died in practice because it lacked lightweight frameworks to successfully implement it. In addition, Agile methodologies as a development philosophy were still a bit immature to successfully drive the development of these services. Huge monolithic projects, largely remained huge monolithic systems. In the end, the time just wasn’t right and the lift was too great.
Fast forward a decade. The modern implementation of microservices, powered by a lightweight messages passing via http with json bodies, coupled with short development sprints of a now mature Agile software development process is allowing new more powerful ways to decouple systems in an iterative way. This time around the technical gains will be huge, as will the social gains (more on this later).
Continuous Integration (CI) and Continuous Deployment (CD) are also enabled by microservices. Since a properly designed microservice implements a small well defined functional contract, exposed only through its external API, it’s easier to build comprehensive automated tests for that specific service. This allows developers to change, test, and release code more quickly and frequently without worrying about the effects on the rest of the system.
Technologies like Docker and Kubernetes are the natural delivery vehicle for microservices. Code as Infrastructure has been around for a while, but the ability to really control the instantiation and discovery of large groups of microservices is a bit of a reach for Hypervisor based VM based cloud systems. When a microservice meets container technology, the level of empowerment for software engineering is pushed to a new level. Using these two concepts architectures will really be living breathing entities, and much more adaptable than in the past.
Social aspects of software engineering are the hidden gain. With the ability to essentially build Lego blocks of code with small understandable functional purpose, it is now possible for an engineer to actually own a piece of code from requirements to deployment and through to production. In large monolithic systems ownership of a ‘part’ was historically impossible. What you don’t own, you care less about. Therefore, if an engineer has ownership and can attach their own personal brand to a part of the system, the quality of that part will be an order of magnitude better.
DevOps is unlocked. DevOps in large part is the social contract engineering has with the broader business. Software is eating the world, and SaaS platforms are on the forefront of the meal. Where SaaS businesses used to have huge Network Operation Centers (NOC’s) to deal with system problems with leagues of humans at the ready to respond to problems, more Agile businesses have recognized that pushing new code into production multiple times a day has literally rendered the NOC useless. In order to support today’s systems, you need to involve the person that wrote the code. NOC’s were built to repetitively solve common problems that by necessity persisted in complex systems. The reality of today’s world is that problems are different everyday and problems that used to wait until the next release, are now fixed the same day and redeployed.
And at this point, the story comes full circle. A person that writes a part of a complex system, attaches his/her brand to that part and is both proud of it and happy to address a problem when it arises. Microservices have an inherent social aspect to them as the entire team is responsible for their behavior. A positive side-effect of this is that the general stress of being “on-call” is much less when the problem being delivered to you, by a product like VictorOps, is inherently known to you; because you know the inner workings of how the code was designed to work.
Every individual technology and part are interesting in their own right. However, the sum of the parts is a very compelling view of the future. A future, where devs are given the power to release quickly and often, and to holistically own their code; from requirements to production. It is my belief that microservices, containers, and above all DevOps will work together to usher in this new reality by bringing it into present day focus.
Special thanks to: Chad Arimura, CEO and Co-founder of Iron.io @chadarimura Eric Knorr, Editor and Chief, @InfoWorld Barunch Sadogursky, Developer Advocate, JFrog @jbaruch Jos Bourmans, VP of Technology Operations, Krux @jiboumans Jordan Dea-Mattson, Chief Architect DevOps PM @jdeamattson Leonid Igolnik, VP of Engineering, CA Technologies Aniket Kulkami, Director of Cloud Engineering, @Box PJ Kimer, CTO and Co-founder, Illumio @pjkirner Mark Nelson, CTO Concur. @MarkTNelson