We at FOSSA have long taken pride in our license scanning technology. For example, the Forrester Wave for Software Composition Analysis recently awarded us the highest score possible in the license risk management criteria. (We were one of only two vendors to earn this distinction.)
But in the interest of making this best-in-class technology even better, our engineering team has been hard at work on an upgraded version of our license scanner.
The previous scanner was very good at what it did, but we identified several areas of improvement for our new version. In this post, we’ll walk you through what changed — and how this will improve your experience with FOSSA.
RELATED: Announcing Our New CLI
A Look Under the Hood
First, because of the way the old scanner detected licenses, we sometimes weren’t able to add licenses or to address errors as quickly as we wanted. That’s because the previous implementation depended on a series of regular expressions to detect licenses, and it was hard to understand how changes to the regular expressions would impact our license scan results.
In the new implementation, we have a directory containing two files for each license known to FOSSA. One file contains the text of the license and the second contains some metadata about the license. We then generate an index from this license that allows us to find licenses quickly and accurately. To add a new license, we add the text of the license and some metadata about the license to that directory and then re-generate the index. If we need to edit the text of the license, we simply make the necessary changes and then re-generate the index. This makes it much faster and safer for us to make changes to our licensing detection.
We also made changes to improve the license’s scanner’s speed — and we have several more in the works. These upgrades don't just make things faster. The speed increase will allow us to do things like adding license scanning capabilities directly to the CLI, which is something that we’re planning to work on in the new year.
What Does This Mean for You?
All of the changes described above happen in the background in our code and on our servers. There’s nothing you need to do. However, you will see the benefits! First and foremost, the new scanner detects more licenses and also produces even fewer false positives when detecting licenses.
Because it's easier to make changes, we will be able to add licenses very quickly after they are updated or created, and we’ll also be more responsive when fixing existing errors in license detection. The new license scanner is better and should keep improving as time goes on.
You might also notice that FOSSA is a bit faster when it's building your dependencies.
The Rollout
We started beta testing the new license scanner in June. We've recently concluded the beta test, and we're beginning to roll out the new technology to all of our customers.
Besides the work on building the new license scanner, we've put a lot of thought into making the rollout as smooth and safe as possible. Before we switch an organization over to the new license scanner, we re-scan all of the dependencies in all of their projects with the new license scanner. This makes sure that when we do switch over, all of your dependencies will be already scanned and you won't see a slow-down in your builds.
It's also easy to switch back to the old license scanner. While the extensive beta testing has made us quite confident in the new license scanner, we also know that software always has bugs in it. We've made switching between the old and new license scanners fast and safe in both directions. We did this by keeping the results from the old license scanner in our system, so switching an organization to the new license scanner or back to the old license scanner is as easy as flipping a switch.
Where are we now in the rollout?
For new users: We're confident that the new license scanner is better, so anybody signing up right now will have the new license scanner enabled.
For current users: We're in the process of re-scanning all currently used dependencies with the new license scanner. Once those re-scans are done, we’ll incrementally roll out the new license scanner out to all of our customers, with multiple quality checks along the way.
The plan is to switch all existing customers to the new license scanner by the end of 2021.
As you can see, we're doing this slowly and carefully, but we also want everyone to see the benefits of the new license scanner as soon as possible.
What Do I Have to Do?
The short answer is that you don't have to do anything: This is a silent improvement to FOSSA's capabilities that doesn't require any action from you. If you see some improvements to license detection, then you may have been switched to the new license scanner already.
If you want to start using the new license scanner right away, send us an email at support@fossa.com or file a ticket at support.fossa.com.
As with all new software, there may be some bugs. If you do notice a misidentified license, a missing license, or another problem with license detection, please let us know by emailing support@fossa.com or filing a ticket at support.fossa.com.
