Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 59 Next »

Many applications incorporate external open source and third-party components that enable developers to build new functionality quickly and efficiently. But while the use of open source components has many benefits, it also introduces risk.

Security Reviewer – Software Composition Analysis (SCA Reviewer) analyzes your application software, identifies project dependencies on 3rd-parties components, and builds a components inventory that lets you track any external piece of code that could be part of your application, directly inside your SDLC, as Jenkins Plugin or using the CLI Interface.

Furher, SCA Reviewer is able to detect:

  • Vulnerable Libraries
  • Vulnerable Frameworks
  • Vulnerable Containers
  • Vulnerable Repositories
  • Discontinued Libraries (abandoned by the community)
  • Obsolete Libraries
  • Poor Man copyrights

Technologies

SCA Reviewer is provided as:

  • Web App, delivered as a group of Docker Images
  • SCA Desktop. .NET Core app for Windows, Linux and macOS
  • CI/CD plugin. Native Jenkins plugin. High experienced CI/CD integrations via CLI 
  • CLI 

Web App

You can view details:

And drill-down to the single issue, a Fixed version is also suggested:

including all References and Exploits:

SCA Desktop

CI/CD plugin

Software Composition Analysis Jenkins and Bamboo native plugins and CLI Interface (test on many CI/CD platforms) provide a 360 degrees solution covering all your DevOps needs. 3rd-party libraries can be analyzed (Open Source Analysis-OSA) using a shared folder located on Network File System (NFS), a Nexus Repository or JFrog Artifactory for discovering Vulnerable Libraries, Vulnerable Frameworks, Blacklisted/ Discontinued/ Outdated / Obsolete/ Deprecated libraries and frameworks. Legal issues like: Blacklisted Licenses, Licenses Conflict, No-licensed libraries, Suspicious (modified) licenses and Poor-man Copyrights are fully-detected from the tool.


Command Line Interface (CLI)

SCA Reviewer offers a very simple CLI interface, suitable for your preferred DevOps integration,for Windows, Linux and macOS

Analyzers

It is able to identify Java, C/C++, Ruby, Groovy, Perl, PHP, JavaScript, TypeScript, nodeJS, Python, Rust, Scala, GO, R, Kotlin, Clojure, ErLang, Shell, PowerShell, LUA and Auto-IT components along with .NET assemblies and Objective-C, Objective-C++, SWIFT support. Once identified, SCA will automatically determine if those components have known, publicly disclosed, vulnerabilities as well as licenses-related issues.

SCA includes the following analyzers:


Analyzer

File Types Scanned

Analysis Method

SRslic **

ASP, ASPX, HTML, JSP, JSF, JAVA, C#, VB.NET, C, CPP, H, HPP, M, MM, SWIFT, PHP, JS, TS, RB, GROOVY, GY, PY, PERL, PL, SCALA, GO, R, KT, CLJ, ERL, SH, PS1, AU3, LUA, XML files

Reveals Licenses in Whitelist, Licenses in Blacklist, License Conflicts, Suspicious Licenses, License Violations and Poor’s man copyrights found in source code.

Artifactory

jFrog Artifactory

Analyzer which will attempt to locate a dependency on a jFrog Artifactory service by SHA-1 digest of the dependency.

Archive

Zip archive format (*.zip, *.ear, *.war, *.jar, *.sar, *.apk, *.nupkg); Tape Archive Format (*.tar); Gzip format (*.gz, *.tgz); Bzip2 format (*.bz2, *.tbz2)

Extracts archive contents, then scans contents with all available analyzers.

Assembly

.NET Assemblies (*.exe, *.dll)

Uses GrokAssembly.exe, which requires .NET Framework or Mono runtime to be installed, otherwise .NET Assemblies will be analyzed by FileInfo and NuSpec analyzers only.

Packrat

CRAN

packrat.lock files (R language).

RetireJS

JavaScript

It uses the manually curated list of vulnerabilities from the RetireJS community along with the necessary information to assist in identifying vulnerable components. Vulnerabilities documented by the RetireJS community usually originate from other sources such as the NVD, OSVDB, NSP, and various issue trackers.

Apache IvyIvy is a dependency manager

Scans dependencies n ivy Ant extension

CMake

CMake project files (CMakeLists.txt) and scripts (*.cmake)

Regex scan for project initialization and version setting commands.

MSBuild

.NET Assembly

Analyzes MSBuild Projects

MavenGradleAnt

Analyze Maven, Ant and Gradle build files for Java

Analyze pom.xml, build.gradle, and build.xml.

GoDep

Analyze GitHub dependency files for GO Language, .go

Analyze vendor.conf, godeps.json, godeps.json gomod files, and gopkg.toml.

Distroless ImagesGoogle Distroless ImagesThey contain only your application and its runtime dependencies

Jar Analyzer

Java archive files (*.jar); Web application archive (*.war)

Examines archive manifest metadata, and Maven Project Object Model files (pom.xml).

NSP

Node Security Project is used to analyze Node.js’ package.json files for known vulnerable packages.

Recently acquired by NPM inc., this service will be still available until September, 30.

SNYK JavaScript, .NET, Java, TypeScript, Python, Ruby, Scala, GO Scans scripts and dependencies

Nuspec

Nuget package specification file (*.nuspec)

Uses XPath to parse specification XML. Analyze also packages.config and (*proj or sln), project.lock.json and project.assets.json or PackageReference.

OpenSSL

OpenSSL Version Source Header File (opensslv.h)

Regex parse of the OPENSSL_VERSION_NUMBER macro definition.

Ruby bundler‑audit

Ruby Gemfile.lock files

Executes bundle-audit and incorporates the results into the dependency-check report.

Autoconf

Autoconf project configuration files (configure, configure.in, configure.ac)

Regex scan for AC_INIT metadata, including in generated configuration script.

CocoaPods

CocoaPods .podspec and podfile.lock files

Extracts dependency information from specification file and lock file, for Objective-C and SWIFT projects.

Composer Lock

PHP Composer and PHP Pear

Parses PHP Pear package.xml, PHP Composer lock and composer.json files for exact versions and dependencies.

Node.js


NPM package specification files (package.json)


Parse JSON format for metadata.


Python Metadata

Python source files (*.py); Package metadata files (PKG-INFO, METADATA); Package Distribution Files (*.whl, *.egg, *.zip) Anaconda and environment.yml

Regex scan of Python source files for setup tools metadata; Parse RFC822 header format for metadata in all other artifacts. Also scans dependencies in yml files.

FileInfo-JarManifest**

jar, war, ear, dll, exe, lib, shared libs and machOS, UPX, PE executables

Reveals Blacklisted Libraries, Outdated Libraries, Other Vulnerable Libraries.

Ruby Gemspec

Ruby makefiles (Rakefile); Ruby Gemspec files (*.gemspec)

Regex scan Gemspec initialization blocks, Rakefile and gemfile.lock for metadata.

SWIFT

SWIFT Package Manager’s Package.swift

Extracts dependency information from swift package file.

**awesome-C, awesome-CPP, cppreference, awesome-dotnet, awesome-javascript, awesome-typescript, SwifterSwift, Three20, PyPi, awesome-scala

Fresh updated lists of best (awesome) libraries, packages and frameworks, specialized for each Programming language

Seeks for new and updated libraries, packages and frameworks coming directly from programmer’s community.

SBT

SCALA

Scans build.sbt for dependencies.

CRAN RScans for add-on packages from CRAN
Akku.scm RScans a Scheme from Akku.scm

CPAN

Perl

Analyze dependencies in Makefile.PL.

LeiningenClojureScans Lein scripts
LuaRocksLUAScans rocks packages

ERL

Erlang HEX

Analyze dependencies in rebar.config.

 RustRustScans cargo.toml file
 au3pmAuto-IT

Scans json.au3 

 AlpineAlpine Linux package keeper (manager)Scans signed tar.gz archives containing programs, configuration files, and dependency metadata
 DebianDebian Package  ManagerScan .deb packages 
 RPMRed Hat Package ManagerScan .rpm files
 BowerJavaScriptScans bower.json
 ChocolateyWindows PackagesScans c4b files
 ClojarsClojureScans Lein scripts and Cloure JAR files
 ConanC/C++ Scans makefile

**Security Reviewer’s SCA unique features

Package Managers

SCA supports several Package Managers.

All Solutions you need

Security Reviewer offers various solutions for Software Composition Analysis:

  • As Security Reviewer integrated module. After Static Analysis completion, a complete application dependency report on 3rd-party libraries/frameworks will be available
  • As Jenkins or Bamboo plugin, it features the ability to perform a Software Composition Analysis build and later view results post build. The SCA Plugin is built using same features that the static analysis plugins offer, including thresholds, charts and the ability to view vulnerability and license information should a dependency have one identified
  • Full-tested integrations with Azure DevOps, Travis CI, Circle CI, TeamCity, CodEnvy, Chef, Ansible, Nexus, AnthillPro, GoCD, among others. Those dependencies can be scan inside your preferred Container System like Docker, Kubernetes, OpenShift, MesoSphere, Rancher, Quay, Singularity, Pivotal CF and any container compliant to APPC specifications

Open source software (OSS) or free and open source software (FOSS) is computer software for which the human-readable source code is made available to the public under license terms permitting the study, modification, and distribution of the software to anyone and for any purpose. The “free” of FOSS refers to the liberty to study, modify and distribute – it does not require that it be made available without cost. The fact that OSS and FOSS can be studied, modified and distributed has created open source communities in which developers continually modify, enhance and troubleshoot the software and contribute these changes back to the community. As a result, the vast majority of enterprises use OSS components in their own proprietary software applications. Doing so brings a host of benefits, including accelerating development time, reducing costs and producing more stable and secure code. While the use of OSS has seen a significant increase over the last 10 years, compliance with the OSS license terms often has not kept up. Many users who download an OSS component may not appreciate that doing so obligates them to abide by a specific software license associated with its use. Unlike commercial software applications or software-as-a-service (SaaS) offerings, click-through acceptance of OSS license terms and conditions are not typically required to gain access to the OSS code. Instead, the license terms are merely referenced on the download site or embedded in a text file within the download package. While there are hundreds of different OSS licenses, they can be broken down into a small subset of license types. They range from: the most permissive licenses, such as Berkley Software Distribution (BSD), MIT and Apache, which permit almost any use of the OSS component and simply limit warranties and disclaim liabilities; to weakly protective licenses, such as the Mozilla Public License (MPL); to the most restrictive “copyleft”-style licenses, such as the GNU General Public License (GPL) or the GNU Lesser General Public License (LGPL). Copyleft licenses can require that any software with which they are packaged must also be distributed under the same copyleft license terms. Use of the more restrictive licenses can therefore have a viral effect, requiring disclosure of, and permitting third parties to use, the source code of one’s own proprietary code. Further, different licenses impose different requirements on modification and distribution of the OSS component. For example, the LGPL license requires users to distribute the source code (or otherwise make it available) if they distribute the OSS component to third parties. Other license types require that users identify changes they have made to the OSS component in a header or other text file included in the distribution package.


The consequences of breach of a commercial software license are relatively clear: the commercial software vendor can sue for damages resulting from the breach of the license termsBut in the OSS context, what are the real risks as a practical matter? Technically, the copyright owners of an OSS component could sue for infringement and/or breach of the OSS license terms. 

While these suits have been filed, given the fact that a particular OSS component could easily have hundreds of contributors and hence hundreds of copyright owners without a common voice, litigation is not typical. Instead, compliance with OSS license terms becomes critical in the context of many important transactions, such as financing and mergers and acquisition transactions. A sophisticated investor or acquirer in any significant financing or M&A (Merge & Acquisition) transaction will always demand a representation and warranty of OSS compliance. Non-compliance creates potential ambiguity around ownership of a material asset and potential post-closing costs of compliance. That ambiguity and associated remediation costs may affect not only the value of the transaction, but also the decision whether to proceed with the transaction at all. How an enterprise manages its use of OSS can speak volumes as to its policies, procedures, structure and culture, all of which are relevant to successful transaction due diligence. To avoid the potential for OSS issues to negatively impact an important transaction, enterprises must develop and follow processes to inventory their use of OSS components, analyze their degree of compliance and remediate any non-compliance long before the term sheet stage of any transaction. Before any enterprise commences an OSS audit, it needs to educate both developers and management on the benefits and risks of incorporating OSS into their proprietary software applications. Asking individuals to participate in an audit where their prior actions could come under a microscope will be far more successful if they understand and appreciate the importance of the outcome.

OSS Component Inventory – Software Bill Of Material (SBOM)

The most significant effort with any OSS audit is inventorying existing use of OSS components. That inventory can occur manually by interviewing each developer and asking them to identify the OSS components they have downloaded and used in the development process. For long standing development teams that have never been through this exercise, that can be a difficult, if not impossible, task. While it may be easy to remember OSS components that were recently incorporated into a development project, that may not be true for software developed years ago. Alternatively, third party software products exist that can automate the inventory process.

I support the following SBOM standards:

Security Reviewer – Composition Analysis module can scan your software code and, using sophisticated pattern matching algorithms, identify the various OSS components present in your software code. While some vendors require that source code be uploaded to their cloud environment for processing, Security Reviewer can operate entirely on-premises and using hashed values of the source code to avoid the risk of source code disclosure outside the enterprise. Unless an enterprise is in the early stages of its development process or has kept an accurate running list of OSS components, the automated process will be far more accurate and complete. Further, in more and more significant financing and M&A transactions, the investors or acquirers themselves are using such automated tools as part of the due diligence process, so to simply assume that the manual inventory process will be “good enough” may be misguided. Representing compliance and possession of an accurate list of OSS components, only to later find out from a counterparty using an automated tool that this is not the case, can be just as bad, if not worse, than not having completed an audit at all.

OSS License Inventory

Whether done manually or via an automated process, the first step is to complete an accurate and complete inventory of each OSS component present in the software. Next, one must identify the license type associated with each OSS component. If the inventory was done manually, then this will require a search of the source website of each OSS component to determine the applicable license type. If the inventory was automated, Security Reviewer automated tool identifies the license type for you, based on its extensive database of OSS components. While there are hundreds of different OSS license types, most OSS inventories tend to consist of only a few dozen license types.

Use Characteristics Inventory

Next, for certain OSS license types, the enterprise must determine how the OSS component is used.

These use characteristics typically include:

Was the OSS component modified? Certain OSS license types require that changes made to the OSS component be identified in a header or other text file. Certain other OSS license types have different distribution requirements if the OSS component was modified. This represent a License Violation.

Is the OSS component distributed to third parties? Certain OSS license types, especially the copyleft style licenses, require that the source code of the OSS component be distributed or made available to third parties.

How was the OSS component incorporated into the enterprise’s code? While this can vary based on different programming language, OSS components can typically be incorporated into a developer’s code in one of three ways:

  • The source code of the OSS component can be copied and pasted into the developer’s source code (as is done with the enterprise’s own proprietary code). Security Reviewer will detect it for you, with accuracy of 10 lines of code and up.
  • The OSS component can be combined with the developed object code at compile time (when the source code is converted into machine-readable code and combined with other components). Security Reviewer will detect it for you.









  • The OSS component can be combined with the developed proprietary code at runtime (when the compiled code is combined with other compiled code and executed to produce the running program).


OSS Compliance Analysis and Implementation

Intellectual property counsel will then review each OSS license against the use characteristics of each OSS component to determine the requisite compliance obligations. Depending upon the license type, those obligations may include:

  • A notice of original copyright and/or attribution to the original author in header files,
    documentation or other user interface elements, such as “about” boxes.
  • A copy of the OSS license in any distribution package to third parties.
  • A copy of the source code of the OSS component in any distribution to third parties.
  • Limitations on use of trademarked names without permission.
  • Identification of changes made to the OSS component.
  • While this may seem like a daunting task, because most OSS components fall under a handful of OSS license types, experienced intellectual property counsel can accomplish the task in less time than one might think. The output from this analysis will typically be a list of recommended actions, such as:
               A complete list of all original copyright and/or attribution notices to include in header files, documentation or other user interface elements, such as “about” boxes.
               A complete copy of all OSS licenses to be included in any distribution to third parties.
               A mechanism to share or make available copies of the source code of certain OSS components.
               A list of trademarked names that should not be used with the software product.
               A complete list of changes made to certain OSS components.
    The recommended actions should then be incorporated into the enterprise’s software development process to ensure compliance with all applicable OSS licenses on an ongoing basis.

OSS License Incompatibility

As part of the OSS audit, it is common to come across certain types of OSS licenses that could be fatal to an enterprise’s future direction or that are simply incompatible with one another. For example, as part of the OSS audit, counsel might discover that a developer has included a certain OSS component licensed under GPL that was incorporated with proprietary code at compile time. Based on the terms of the GPL license, the entire software package must be icensed under the GPL license and ALL of the software package’s source code, including the proprietary code, must be disclosed to third parties. For most software companies, the source ode of proprietary software is the crown jewel of the enterprise and any required disclosure could significantly reduce the value of the company. In addition, there are certain OSS license types that are inherently incompatible with others. For example, an OSS component licensed under MPL combined with an OSS component licensed under GPL cannot be distributed without violating the terms of one of the licenses. In either situation, the OSS component causing the compliance issue will typically need to be replaced with another OSS component with a compatible license or rewritten with proprietary code. The potential that this remediation work will be required is a prime reason to commence and compete an OSS audit well in advance of a significant transaction.

Implement an Open Source Policy

After the initial OSS audit is completed and the enterprise is in compliance with all of the licenses applicable to the OSS components used in its software products, management should implement an Open Source Policy. In its simplest terms, an Open Source Policy is a set of written rules that governs the management of OSS (both use of and contribution to) with detailed specifications as to how an enterprise will implement these rules on a daily basis. That policy should include a list of banned OSS license types that developers can reply upon when choosing whether to use and incorporate a specific OSS component. Any OSS license types not on the banned list should be reviewed with intellectual property counsel prior to their use. As part of the policy, the OSS inventory report created from the initial audit should be continually updated as a living document. The policy should also incorporate periodic training of OSS considerations for developers. This will keep an open dialogue between developers and intellectual property counsel as new OSS license types emerge and existing OSS licenses are tested in the court systems.


External Data Sources


Security Reviewer - Software Composition Analysis (SCA) discovers Vulnerable, Blacklisted, Discontinued by the Community, Deprecated and Obsolete Libraries/Frameworks. This is achieved by continuous monitoring of the following external data sources:

-- Debian Security https://security-tracker.debian.org
-- Linux Security https://linuxsecurity.com/
-- RedHat Security https://access.redhat.com/security
-- Oracle Security Advisory https://www.oracle.com/technetwork/security-advisory
-- SuSe OVAL Descriptions https://www.suse.com/support/security/oval/
-- Ubuntu CVE Tracker https://people.canonical.com/~ubuntu-security/cve/main.html
-- Alpine Linux Security https://alpinelinux.org
-- CentOS Security https://www.centosblog.com/categories/security/
-- Microsoft Security Response Center https://portal.msrc.microsoft.com/en-us/security-guidance
-- OSS Index by Sonatype: https://ossindex.sonatype.org/
-- NVD by NIST: https://www.nist.gov/programs-projects/national-vulnerability-database-nvd
-- VulnDB: https://vuldb.com/
-- Maven Central: https://mvnrepository.com/repos/central (Java, Scala, Kotlin)
--  GitHub Advisory Database (RubyGems) 
-- GitHub Advisory Database (Erlang)
-- Common Weakness Enumeration (CWE): http://cwe.mitre.org/
-- CVE Details: https://www.cvedetails.com/
-- Aqualogic Trivy: https://github.com/aquasecurity/trivy-db


SCA uses the following community-driven 'Awesome' lists too:
-- Awesome Java https://github.com/akullpp/awesome-java
-- Awesome .NET https://github.com/quozd/awesome-dotnet
-- Awesome Android https://github.com/JStumpp/awesome-android
-- Awesome C libraries https://github.com/kozross/awesome-c
-- Awesome C++ https://github.com/fffaraz/awesome-cpp
-- Awesome JavaScript https://github.com/sorrycc/awesome-javascript
-- Awesome TypeScript https://github.com/dzharii/awesome-typescript
-- Awesome Python https://awesome-python.com/ and https://pythonawesome.com
-- Awesome Ruby https://github.com/markets/awesome-ruby
-- Awesome Scala https://github.com/lauris/awesome-scala
-- Awesome GO https://github.com/avelino/awesome-go
-- Awesome PHP https://github.com/ziadoz/awesome-php
-- Awesome Swift https://github.com/matteocrippa/awesome-swift
-- Awesome iOS https://github.com/vsouza/awesome-ios
-- Awesome Kotlin https://github.com/KotlinBy/awesome-kotlin
-- Awesome Groovy libs https://github.com/kdabir/awesome-groovy
-- Awesome shell scripts https://github.com/alebcay/awesome-shell
-- Awesome R https://awesome-r.com
-- Awesome PowerShell https://github.com/janikvonrotz/awesome-powershell
-- Awesome Auto-It https://github.com/J2TeaM/awesome-AutoIt
-- Awesome LUA https://github.com/LewisJEllis/awesome-lua
-- Awesome Clojure https://github.com/razum2um/awesome-clojure
-- Awesome Erlang https://github.com/drobakowski/awesome-erlang
-- Awesome Rust https://github.com/rust-unofficial/awesome-rust

Further, SCA uses the libHunt service:
-- Java https://java.libhunt.com/
-- .NET https://dotnet.libhunt.com
-- Android https://android.libhunt.com/
-- C/C++ https://cpp.libhunt.com/
-- JavaScript/TypeScript https://js.libhunt.com/
-- Python https://python.libhunt.com
-- Ruby/Groovy https://ruby.libhunt.com/
-- Scala https://scala.libhunt.com/
-- GO https://go.libhunt.com/
-- PHP https://php.libhunt.com/
-- Swift https://swift.libhunt.com/
-- iOS https://ios.libhunt.com/
-- Kotlin https://kotlin.libhunt.com/
-- Rust https://rust.libhunt.com/ 

Regarding the license/legal issues, like Blacklisted Licenses (Strong CopyLeft included), License Conflict, License Violations, Suspicious Licenses (modified and missed licenses), Poor-man CopyRight, SCA uses the following external data sources:
-- SPDX: https://spdx.org/
-- Open Source Initiative: https://opensource.org/licenses
-- GNU compatible license list: http://www.gnu.org/licenses/license-list.html
-- Creative Commons: https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/
-- Comparison of FOSS licenses: https://en.wikipedia.org/wiki/Comparison_of_free_and_open-source_software_licenses
-- FLOSS license chart: https://dwheeler.com/essays/floss-license-slide.html








  • No labels