Are Your Open Source Policies Slowing Down Innovation?

For many engineers to take advantage of open source software, they have to go through a very manual and slow process of filling out request forms before building anything dependent on open source packages. For the most part, we think this process is damaging to both the engineering culture and the speed of innovation.

Long story short, modern software development best practices coupled with the scale and frequency at which open source is leveraged make it impossible to maintain a request queue and take advantage of the market opportunity at a competitive rate. It’s no longer a solution that makes sense.

Evolving Software Best Practices

Modernization of Software Development

Over the past decade, the Agile Methodology is increasingly adopted as the preferred methodology as opposed to the waterfall (traditional) approach to software development. Instead of planning an entire project's trajectory up front (the waterfall method), teams are now operating in sprints. Each sprint cycle allows new information (from customers, the market, technical issues) to be incorporated into the evolving product design in order to hit the end project goals in the best way possible. This adaptability (or agility) can be a great method for delivering the best product in a timely manner. However, it does make it hard to plan which open source components you might need far in advance.

The shift in methodology is also reflected in the shift from a quarterly/annual release to a continuous deployment process (CI/CD). With the waterfall development method, it made sense to do one big release at the end of a project. However, with the Agile Method, a more continuous deployment process is needed.

These shifts in best practices for software development are accelerated by the shifting landscape. The rise of SaaS and cloud-based products means that improvements can be made rapidly - without requiring customers to download another version. Not only that, an ecosystem of build tools such as TravisCI, Jenkins, and CircleCI have emerged to aid in continuous integration/continuous delivery. These tools help companies maintain quality while accelerating product innovation with continual updates and improvements to their software.

Exponential Open Source Adoption

There has also been an exponential increase in the adoption of Open Source Software (OSS) across almost every industry. OSS used to help develop almost every piece of software, from developing software to make planes fly and manufacturing plants run to the mobile apps on your phone. Looking at the exponential adoption of OSS, it’s clear that it offers immense benefits to companies from reducing total cost of ownership, to getting access to industry innovations, to developing more secure and higher quality products. The number of open source packages available has also increased.

The Outcomes of a Manual OSS Approval Process

With the shifting landscape of software development, there are three common outcomes when relying on a manual OSS approval process:

Missed Product Launch Deadlines

The simplest reason for avoiding open source request forms - and the open source request process - is that it takes time to submit, review, and approve open source packages. This time can divert resources away from key initiatives - like building a deal-closing feature or patching a security vulnerability - or it can push back a project deadline. This may be irrelevant for one or two packages, but the scale in which open source is being used has a compounding impact across product development as a whole, causing missed market opportunities or delays in revenue generating product releases.

Lack of Developer Compliance With Key Processes

Although missed market opportunity is the most damaging aspect of restrictive open source practices, the reality of enforcing compliance - especially if you are relying on a manual system - is near impossible. Often the accurate number of direct dependencies (components developers are intentionally adding) is between 5-10x lower than the total number of dependencies. With the excitement to build impactful new products and the pressure to hit roadmap deadlines, compliance is usually the last thing on the developer’s mind. Requesting usage may disrupt flow or cause significant delays. With the proliferation of open source components, it may actually just be too difficult to cite all of the dependencies used when logging by hand. Developers can feel that the company doesn’t care about best practices or innovative solutions and look for alternative jobs that will let them move at the pace they desire, leveraging the tools they want to use. The more disruptive, difficult, or delaying a process is, the more likely people will not adhere to a policy with any rigor.

Lack of Accuracy

Even if your engineering team complies perfectly, the odds are that your data is incorrect. Manual processes are prone to human error. While it may seem simple to request a specific component, in practice each component is often dependent on other open source components, which is often dependent on other open source components, and so on. When an engineer requests to use package A, they may honestly not know that package B, C, and D are also going to be included in your production code. The dependency tree can be several layers deep, yet your company is held liable for all of the licenses and security vulnerabilities, even if you are leveraging an open source component indirectly.

How Can You Streamline Open Source Usage And Maintain Compliance?

Educate Developers on Licenses

Many companies include open source compliance in their developer onboarding. Understanding which licenses to look for and which to avoid can help developers evaluate which packages to use. There are tons of resources you can provide to developers, like tldrlegal, that help translate license text and legal jargon into plain English. Another resource you can provide is a means for developers to search already approved packages.

Even more important is developing an understanding of why licensing is important to both developers and the company. Key reasons that may resonate with a developer include:

  • Being a good member of the open source involves being compliant and using open source software the way the author intended
  • Being able to integrate with new marketplaces and partners like the Google Cloud Platform, which can require proof of open source compliance
  • Enterprise companies often require an open source due diligence report before buying new software or deploying on-prem software
  • Due diligence for company events such as raises, acquisitions, and IPOs

Implement Continuous Compliance

Implementing continuous compliance is another way to map to the developers' workflow. Continuous compliance means evaluating open source compliance at every commit - accelerating innovation along with CI/CD build tools. When evaluating continuous compliance you should consider the following features:

Integrations with the CI/CD systems

Integrations are key to continuous compliance. Tools should integrate with developer systems and should seamlessly scan code without requiring developers to adjust their workflows. Depending on your company’s policies, they should also be able to fail merges to production when the open source components used do not meet company policy.

A Command Line Interface Agent (CLI)

Ensure the compliance tool you select can be operated from the command line. This minimizes workflow interruptions, improving engineering productivity. You should be able to trigger scans and run tests against your policy without leaving the CLI.

Deep Dependency Scanning

Ensure you are scanning all of the open source packages used, including deep and transitive dependencies. This will improve accuracy, and should show your engineering team exactly where in the dependency tree (map of how all the open source components are connected) the compliance issues lie.

By automating continuous compliance not only are you reducing workflow interruptions, but you are enhancing developer productivity without increasing the company’s risk profile.