travis: add scancode job

The goal: check license offenders in pull request

This is similar to what astyle does in Travis. We get list of files being changed. Because scancode does not support list of files being scanned but rather a file or directory, we copy files to SCANCODE folder. Execute scancode license check in this folder and check for offenders.

The rules there are: code files must have a license and SDPX identifier. If they don't, we print these and ask for review.

Author: 0xc0170

.travis.yml

@@ -55,10 +55,21 @@ matrix:
     - <<: *basic-vm
       name: "license check"
       env: NAME=licence_check
+      language: python
+      python: 3.6.8 # scancode-toolkit v3.1.1 requires v3.6.8
+      install:
+        - pip install scancode-toolkit==3.1.1
+      before_script:
+        - mkdir -p SCANCODE
       script:
-        - |
-          ! grep --recursive --max-count=100 --ignore-case --exclude .travis.yml \
-          "gnu general\|gnu lesser\|lesser general\|public license"
+        # scancode does not support list of files, only one file or directory
+        # we use SCANCODE directory for all changed files (their copies with full tree)
+        - >-
+            git diff --name-only --diff-filter=d FETCH_HEAD..HEAD \
+              | ( grep '.\(c\|cpp\|h\|hpp\)$' || true ) \
+              | while read file; do cp --parents "${file}" SCANCODE; done
+        - scancode -l --json-pp scancode.json SCANCODE
+        - ./tools/test/travis-ci/scancode.py -f scancode.json
 
     - <<: *basic-vm
       name: "include check"

tools/test/travis-ci/scancode.py

@@ -0,0 +1,62 @@
+"""
+Copyright (c) 2020 Arm Limited. All rights reserved.
+
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations 
+"""
+
+import json
+import argparse
+import sys
+
+def license_check(file):
+
+    offenders = []
+
+    # find all licenses in the files, must be licensed and permissive
+    with open(file, 'r') as scancode_output:
+        results = json.load(scancode_output)
+
+        for file in results['files']:
+            # ignore directory, not relevant here
+            if file['type'] == 'directory':
+                continue
+            if not file['licenses']:
+                offenders.append(file)
+            for i in range(len(file['licenses'])):
+                if (not file['licenses'][i]['spdx_license_key'] or (file['licenses'][i]['category'] != 'Permissive')):
+                    offenders.append(file['path'])
+
+    if offenders:
+        print("Found files with missing license details, please review and fix")
+        print(*offenders, sep = ", ")
+        sys.exit(-1)
+    else:
+        sys.exit(0)
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="License check.")
+    parser.add_argument('-f', '--file',
+                        help="scancode-toolkit output json file")
+    return parser.parse_args()
+
+if __name__ == "__main__":
+
+    args = parse_args()
+
+    if args.file: 
+        license_check(args.file)
+    else:
+        sys.exit("Missing scancode json file to be checked")