v32.4.0

[AyanSinhaMahapatra]

This is a feature release with:

• python 3.13 support
• support added for adding required phrases to rules automatically
• misc license and package detection improvements
• new and updated license detection rules and new licenses
• misc bugfixes, dependency and documentation updates

There are new data attributes, and we have a output format version bump
from 4.0.0 to 4.1.0. The changes in Output Data Structure are:

• A new resource level attribute sha1_git is added, which has
  the corresponding checksum value for files, and is empty for
  directories. This is returned optionally with the --info plugin.

• A new resource level attribute is_community is added, which is
  True from commonly used files used for community/project maintainence.
  This is returned optionally with the --classify plugin.

These are the details for the most important changes introduced::

• Add support for adding required phrases in rules automatically using
  some console scripts and CLI options using already marked required
  phrases for the same license-expression and license field attributes
  The new console scripts are:

  • add-required-phrases to add required phrases from other rules or
    license attributes
  • gen-new-required-phrases-rules to add required phrase rules for
    marked required phrase in rules
    This improves detection accuracy and reduces false positives.
    Update rules with required phrases automatically #3924
    Improve license required phrase generation #4237
    Refine required phrases with stopwords #4238 #4241

• Default value of processes used for scancode scans is changed from
  1 to N-1, where N is the number of CPU processes available in the
  system. change default value of --process to (number of CPUs)-1 #4104

• Also return sha1_git checksums for each files with --info plugin.
  Add Git blob sha1 to checksums returned with info #624

• Equivalent words like license and licence, as well as plurals are
  now treated as the same in license detection. With this,
  many redundant rules have been deprecated.
  Support equivalent words in license detection #4190 #4215

• Support running scancode with python3.13
  Update and use latest native dependencies with py3.13 support,
  update and test py3.13 usage in CI and other scripts, and
  update other third-party dependencies, use latest skeleton
  Support python 3.13 with updated dependencies #4430

• Misc license detection improvements, new licenses and license
  detection rules.
  Improve maven license detection #4261
  Improve license detection #4412
  Improve npm license detection #4405
  Update licenses and add rules #4278
  Update various license rules #4093

• Fix an issues where pip install scancode-toolkit was failing
  because of a compatibility issue with Click
  Fix click compatibility issues with commoncode v32.3.0 #4427

What's Changed

• 4181 update about to use license expression by @chinyeungli in 4181 update about to use license expression #4184
• Refine postgresql RULE by @alok1304 in Refine postgresql RULE #4111
• Update rules with required phrases automatically by @AyanSinhaMahapatra in Update rules with required phrases automatically #3924
• Add new rule for eupl license by @alok1304 in Add new rule for eupl license #4204
• Add required phrase markers to CC license rules by @dotarjun in Add required phrase markers to CC license rules #3644
• Fix minor message typo by @pombredanne in Fix minor message typo #4228
• Add tests for all PyPI METADATA versions (Ensure all versions of pypi METADATA parsing is tested #4175) by @alok1304 in Add tests for all PyPI METADATA versions (#4175) #4180
• Drop Ubuntu 20 by @pombredanne in Drop Ubuntu 20 #4240
• Fix false positive detection heuristics by @alexzurbonsen in Fix false positive detection heuristics #4009
• Add autosar license rules by @pombredanne in Add autosar license rules #4242
• Add new rule for mit by @alok1304 in Add new rule for mit #4121
• Update various license rules by @AyanSinhaMahapatra in Update various license rules #4093
• Add new license rules for new Elasticsearch notices by @NucleonGodX in Add new license rules for new Elasticsearch notices #4041
• Refine required phrases with stopwords License stopwords problems #4238 by @pombredanne in Refine required phrases with stopwords #4238 #4241
• Ensure opam tests are running by @pombredanne in Ensure opam tests are running #4271
• Support equivalent words in license detection False Positive Detection of LGPL-2.0-plus and Other Licenses #4190 by @pombredanne in Support equivalent words in license detection #4190 #4215
• Improve maven license detection by @pombredanne in Improve maven license detection #4261
• Improve license required phrase generation by @pombredanne in Improve license required phrase generation #4237
• Fix and enhance support for different bazel metadata versions by @abraemer in Fix and enhance support for different bazel metadata versions #4194
• Add DUMB License (Consider adding new "dumb" license (or adding a rule) #4058) by @alok1304 in Add DUMB License (#4058) #4143
• Add test for false positive GPL3 license by @alok1304 in Add test for false positive GPL3 license #4106
• Update test_detect.py by @pombredanne in Update test_detect.py #4311
• Update licenses and add rules by @AyanSinhaMahapatra in Update licenses and add rules #4278
• Add test for equivalent word by @alok1304 in Add test for equivalent word #4305
• Display extra-words in detection_log if present by @alok1304 in Display extra-words in detection_log if present #4402
• fix: change version number in field "key" for code-credit-license-1.0-0 by @leoreinmann in fix: change version number in field "key" for code-credit-license-1.0-0 #4416
• fix: change version number in field "name" for license "LiLiQ-R-1.1" by @leoreinmann in fix: change version number in field "name" for license "LiLiQ-R-1.1" #4418
• Fix click compatibility issues with commoncode v32.3.0 by @AyanSinhaMahapatra in Fix click compatibility issues with commoncode v32.3.0 #4427
• Return sha1_git checksum for all files as info by @AyanSinhaMahapatra in Return sha1_git checksum for all files as info #4425
• Improve license detection by @AyanSinhaMahapatra in Improve license detection #4412
• Improve npm license detection by @AyanSinhaMahapatra in Improve npm license detection #4405
• Documentation enhancement by @chinyeungli in Documentation enhancement #4448
• change default value of --process to (number of CPUs)-1 by @xsuchy in change default value of --process to (number of CPUs)-1 #4104
• Replace broken link for Sun Public License SPL 1.0 by @alok1304 in Replace broken link for Sun Public License SPL 1.0 #4109
• Support python 3.13 with updated dependencies by @AyanSinhaMahapatra in Support python 3.13 with updated dependencies #4430

New Contributors

• @alok1304 made their first contribution in Refine postgresql RULE #4111
• @dotarjun made their first contribution in Add required phrase markers to CC license rules #3644
• @NucleonGodX made their first contribution in Add new license rules for new Elasticsearch notices #4041
• @abraemer made their first contribution in Fix and enhance support for different bazel metadata versions #4194
• @leoreinmann made their first contribution in fix: change version number in field "key" for code-credit-license-1.0-0 #4416

Full Changelog: v32.3.3...v32.4.0

---

This discussion was created from the release v32.4.0.