All Things Open interview with Jeff Luszcz, Palamida
Does your company know how much open source it uses?
Licenses are the legal underpinning of open source projects, but companies don't always know how to manage them. Jeff Luszcz founded Palamida to help organizations ensure they were complying with upstream software licenses. Along the way, he and his team discovered that being unaware of the open source licenses in use leads to being unaware of vulnerabilities that need to be patched.
At All Things Open, Jeff is presenting two talks: How to realize the business and security ROI for managing open source and How to protect your Linux based products from IP violations and security vulnerabilities. I recently spoke to him about the intersection of licensing and security. Find out more in this interview.
At first glance, intellectual property violations and security vulnerabilities seem like unrelated topics. How did they end up in the same talk?
When Palamida first started in 2004, the vast majority of our customers were concerned about making sure they were respecting the licensing obligations that came with the open source software (OSS) components they were using. They were (and still are) shocked to learn that they undercounting their use by over 95% in the typical case. This leads to intellectual property (IP) and licensing violations and massive amounts of rework to replace out of policy components.
Along the way, security vulnerabilities were discovered to be an issue as well with the same root cause. If a company wasn't tracking open source code for the purposes of IP compliance, they also weren't paying attention to version updates, bug reports, and patches. This leads to an installed base that contains sometimes decade old versions of libraries with known security issues.
Heartbleed (the security vulnerably found in OpenSSL in 2014) is still found during Palamida scans even today. This is even after a worldwide push to find and upgrade this library. Most software products have not been scanned or reviewed and have vulnerable software components sitting there silently.
How can grassroots employees convince their management that using and contributing to open source projects is good for the company?
This can be a long struggle. It was shocking to hear that OpenSSL only received a few thousand dollars a year in donations before Heartbleed. Post-Heartbleed, they are doing much better. Other projects are not as lucky and still struggle with financial and source contributions.
One thing that I have seen help is to point to the important open source projects that power your product. These are direct replacements for homegrown code and be pointed to as a "cost savings." Keeping those components alive and thriving helps your company's bottom line as well. The two "easy" onramp activities that companies do are financial donations and bug reports/bug fixes. Donating money is quick, easy, and doesn't affect deadlines. Even a modest donation would be appreciated.
If your company's beer or coffee budget is larger than your open source contribution budget, I'd say you have a problem.
Donating code or time can be harder. The larger your company is, the more legal hoops you may need to jump through. A bug fix or patch is often less controversial than donating a large feature or full-time developer. This is a great way to start and to help the management chain learn the ins and outs of working with open source contributions.
More and more developers are making open source contributions a part of their job description and are using their influence in the company to make sure that they have the support required to push code upstream. "If I can't work on Linux or Hadoop, I'll go someplace I can." While that might not work for many developers, if you are lucky enough to be at this stage of your career, you can greatly affect your company.
Your talks seem like they will focus on managing from where the open source companies are using has come from. What role can organizations like the Software Freedom Conservancy and the Apache Foundation play in helping with this?
These groups represent influential creators of open source projects and put a face on what many see as two ends of a philosophical spectrum around open source licensing (copyleft and attribution). These groups help define the expectations around the creation and use of open source. We often tell our customers to not only look at the license obligations but the philosophy and writings of the authors. It's one thing to be legally compliant with a license; it's another to fit in to the culture of the group you are working with. Sometimes companies will try "legally accurate" ways to comply with the text of a license, but completely break the spirit of a license. The community will quickly show you that this is unacceptable.
While the onus is on the users of open source to correctly use open source, it's in the best interests of groups like these to encourage proper behavior. Most companies want to do the right thing, but often lack the knowledge of what to do and how to start.
By creating training, working with developers both inside and outside of their communities, and making it easy for companies to communicate with the organization, these types of groups can help make open source easier for companies to work with.
What do you see as the biggest challenge in open source compliance, and how does the future look for it?
No one believes how much open source they are actually using. When we work with companies we ask them to disclose how much open source they are using. The most common answer is, "We know we are using open source, but we don't know where." Companies typically only can account for 5% of their true open source usage when pressed. A review of their code will find hundreds (and sometimes thousands) of libraries that didn't know they were using. Each of these missed libraries represents a license violation.
Getting the engineering management chain to understand and correctly budget for compliance can be difficult until they see the results of an audit. It becomes easier to "do it right the first time" after a team needs to remove dozens of libraries and create and publish license disclosures for hundreds more and the potential security holes are closed by upgrading and patching as required. It's always cheaper from a cost and reputation perspective to build something correctly from the ground up instead of retrofitting after the fact.
The future is arriving quickly. Many companies were brought kicking and screaming into compliance post-Heartbleed when they discovered how little knowledge of their open source usage existed. When every CEO asked what they thought was going to be a quick question—"Where are we using OpenSSL?"—turned into a resource intensive multi-month analysis, project people started to pay attention. This event has really changed companies' compliance efforts as they realized how far off the mark their awareness was.
What All Things Open talks are can't-miss for you this year?
I have an interest in open source licensing, and this year's All Thing Open conference has one of the best groups of talks on that topic that I've seen in a while.
I've worked with Heather Meeker for many years and I'm looking forward to her Setting your code free (Without scaring the lawyers): Licensing and IP considerations when doing code releases and contributions. Heather's done a ton of this type of work and will have some great stories of both the best and worst ways to handle this process. I always learn something new at her talks.
Also, Bradley Kuhn's GPL enforcement is good for business talk is certainly going to be a great talk to understand the goals and motivations of the community who uses the General Public License for their open source. Unfortunately it's at the same time as my talk, so I won't be able to see it this year!
Lastly, the Engineering, business, and legal choices for bringing commercial software to open source and Understanding open source licenses talks both look interesting. I may need to split myself in two to catch both.