There has been a seismic shift in software development over the past 25 years that I have been developing software. The single biggest factor contributing to this change is the growth of open source software. Before open source, software developers had access to very few pre-built, high-level software components and libraries. There were a smattering of lower-level libraries, such as the C++ standard libs and Rogue Wave’s C++ tools, that abstracted away some of the low level details of common data structures, and communication protocols, and these were definitely useful. And, of course, there were a handful of commercial relational database offerings and application server platforms that were a staple of any large software system. But for the most part, development teams had to build the software for an application from scratch.
However, the advent of the open source movement changed the rules of the game. It was now possible for developers to leverage the work done by others, rather than re-inventing the wheel over and over again. The other, possibly more important benefit is that because the source code is open, it can be reviewed and improved by the community (often including experts in a particular field) and everyone benefits. And if a particular open source package doesn’t quite solve your problem, you can change it to suit your needs, without having to build the entire package yourself.
Many software developers have never worked in a world without open source, and even those of us that have, often tend to take the availability of open source for granted. However, if I reflect on the current state of the software development world, it makes me feel like there has never been a better time to build software. At the risk of overstating or misusing a metaphor, I would almost call this the Golden Age of Software Development. There are now excellent open source tools and projects that span the entire gamut of software development including source code management, software development process, virtual hardware abstraction and management, databases (SQL, NoSQL, Big Data, etc), network based application frameworks, and on and on. If you need to implement a particular algorithm or service, there is very likely an open source package that you can use either “out of the box” or at least as a basis to start with. In addition, there are usually multiple alternatives for most of these components, so you can choose the one that fits your team/organization’s desired process the best.
I think it is pretty clear that leveraging open source software has huge potential benefits in developer productivity and time to market for large software projects. However, this easy access to many “shiny objects” can call to a developer like the Sirens of Greek mythology. The danger is that each time you introduce new open source code into your project, you need to make sure that you can maintain and troubleshoot that piece of software. Having access to the source code can help, because theoretically you can fix bugs yourself, but if it is a complex piece of software, that may not be practical given your existing time/resource constraints. The more components you introduce, the higher the likelihood that you will eventually encounter problems with one or more of them.
So while I am a huge proponent of open source, and we leverage it extensively at VictorOps, I also think that it is important to evaluate each component we want to incorporate with these thoughts in mind:
• How mature is the code? The latest, trendy, cutting-edge code/technology can be great, as long as the possible cuts don’t cause your application to bleed out. I like to make sure that key open source components have been proven in production with some larger volume users for a while before adopting them. Let other early adopters work out the kinks so that you don’t have to spend a lot of time troubleshooting issues, or worse, find out after the code is in production that it is too unreliable to support.
• Is there support (and good documentation) available for the code? This can include an active open source community and/or commercial support. In my opinion, the biggest advantage of open source is not that it’s free. It might be reasonable to pay for support on key components of your infrastructure so that you know you can call someone in the middle of the night to help fix a problem.
• Keep the number of technologies manageable. Sometimes there is a tendency to want to try out a new technology even though your existing solution works well. This can often lead to a proliferation of different technologies in your infrastructure that all perform the same function, with the idea that you will refactor to use the best one of the alternatives later. Usually, later never comes and you are left supporting all of them. Resist the urge to hop from technology to technology and instead spend time evaluating the best alternatives and introducing them when you have time to do it right.
Open source created a revolution in software development and has helped fuel the ever-increasing pace of innovation in the technology industry. There are some important decisions that need to be made when choosing open source, and many of these are dependent on the requirements of the software you are building, but the opportunity it provides to create complex software and services that will benefit users more quickly makes this an exciting time to be a software developer.