Skip to content

Create and upload ssdlc_compliance_report.md #1405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 52 additions & 14 deletions .evergreen/.evg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,45 @@ functions:
content_type: ${content_type|text/plain}
display_name: "orchestration.log"

"create and upload SSDLC release assets":
- command: shell.exec
shell: "bash"
params:
working_dir: "src"
env:
PRODUCT_NAME: ${product_name}
PRODUCT_VERSION: ${product_version}
script: .evergreen/ssdlc-report.sh
- command: ec2.assume_role
params:
role_arn: ${UPLOAD_SSDLC_RELEASE_ASSETS_ROLE_ARN}
- command: s3.put
params:
aws_key: ${AWS_ACCESS_KEY_ID}
aws_secret: ${AWS_SECRET_ACCESS_KEY}
aws_session_token: ${AWS_SESSION_TOKEN}
local_file: ./src/build/ssdlc/ssdlc_compliance_report.md
remote_file: ${product_name}/${product_version}/ssdlc_compliance_report.md
bucket: java-driver-release-assets
region: us-west-1
permissions: private
content_type: text/markdown
display_name: ssdlc_compliance_report.md
- command: s3.put
params:
aws_key: ${AWS_ACCESS_KEY_ID}
aws_secret: ${AWS_SECRET_ACCESS_KEY}
aws_session_token: ${AWS_SESSION_TOKEN}
local_files_include_filter:
- build/ssdlc/static-analysis-reports/*.sarif
local_files_include_filter_prefix: ./src/
remote_file: ${product_name}/${product_version}/static-analysis-reports/
bucket: java-driver-release-assets
region: us-west-1
permissions: private
content_type: application/sarif+json
display_name:

"upload test results":
- command: attach.xunit_results
params:
Expand Down Expand Up @@ -825,24 +864,21 @@ functions:
params:
working_dir: "src"
script: |
tag=$(git describe --tags --always --dirty)

# remove the leading 'r'
version=$(echo -n "$tag" | cut -c 2-)

cat <<EOT > trace-expansions.yml
release_version: "$version"
EOT
cat trace-expansions.yml
PRODUCT_VERSION="$(echo -n "$(git describe --tags --always --dirty)" | cut -c 2-)"
cat > ssdlc-expansions.yml <<EOF
product_version: "$PRODUCT_VERSION"
product_name: "${product_name}"
EOF
cat ssdlc-expansions.yml
- command: expansions.update
params:
file: src/trace-expansions.yml
file: src/ssdlc-expansions.yml
- command: papertrail.trace
params:
key_id: ${papertrail_key_id}
secret_key: ${papertrail_secret_key}
product: ${product}
version: ${release_version}
product: ${product_name}
version: ${product_version}
filenames:
- "src/build/repo/org/mongodb/*/*/*.jar"
- "src/build/repo/org/mongodb/*/*/*.pom"
Expand Down Expand Up @@ -1580,15 +1616,17 @@ tasks:
- func: "publish snapshot"
- func: "trace artifacts"
vars:
product: mongo-java-driver-snapshot
product_name: mongo-java-driver-snapshot
- func: "create and upload SSDLC release assets"

- name: publish-release
git_tag_only: true
commands:
- func: "publish release"
- func: "trace artifacts"
vars:
product: mongo-java-driver
product_name: mongo-java-driver
- func: "create and upload SSDLC release assets"

- name: "perf"
tags: ["perf"]
Expand Down
63 changes: 57 additions & 6 deletions .evergreen/ssdlc-report.sh
Original file line number Diff line number Diff line change
@@ -1,15 +1,66 @@
#!/bin/bash
#!/usr/bin/env bash

set -o errexit
set -eu

# Supported/used environment variables:
# PRODUCT_NAME
# PRODUCT_VERSION

if [ -z "${PRODUCT_NAME}" ]; then
echo "PRODUCT_NAME must be set to a non-empty string"
exit 1
fi
if [ -z "${PRODUCT_VERSION}" ]; then
echo "PRODUCT_VERSION must be set to a non-empty string"
exit 1
fi

############################################
# Main Program #
############################################
RELATIVE_DIR_PATH="$(dirname "${BASH_SOURCE[0]:-$0}")"
source "${RELATIVE_DIR_PATH}/javaConfig.bash"

echo "Creating SSLDC reports"
printf "\nCreating SSDLC reports\n"

declare -r SSDLC_PATH="${RELATIVE_DIR_PATH}/../build/ssdlc"
declare -r SSDLC_STATIC_ANALYSIS_REPORTS_PATH="${SSDLC_PATH}/static-analysis-reports"
mkdir "${SSDLC_PATH}"
mkdir "${SSDLC_STATIC_ANALYSIS_REPORTS_PATH}"

printf "\nCreating SpotBugs SARIF reports\n"
./gradlew -version
./gradlew -PssdlcReport.enabled=true --continue -x test -x integrationTest -x spotlessApply clean check scalaCheck kotlinCheck testClasses || true
echo "SpotBugs created the following SARIF files"
find . -path "*/spotbugs/*.sarif"
set +e
# This `gradlew` command is expected to exit with a non-zero exit status,
# because it reports all the findings that we normally explicitly exclude as "No Fix Needed"/"False Positive".
./gradlew -PssdlcReport.enabled=true --continue -x test -x integrationTest -x spotlessApply check scalaCheck kotlinCheck
set -e
printf "\nSpotBugs created the following SARIF reports\n"
IFS=$'\n'
declare -a SARIF_PATHS=($(find "${RELATIVE_DIR_PATH}/.." -path "*/spotbugs/*.sarif"))
unset IFS
for SARIF_PATH in "${SARIF_PATHS[@]}"; do
GRADLE_PROJECT_NAME="$(basename "$(dirname "$(dirname "$(dirname "$(dirname "${SARIF_PATH}")")")")")"
NEW_SARIF_PATH="${SSDLC_STATIC_ANALYSIS_REPORTS_PATH}/${GRADLE_PROJECT_NAME}_$(basename "${SARIF_PATH}")"
cp "${SARIF_PATH}" "${NEW_SARIF_PATH}"
printf "%s\n" "${NEW_SARIF_PATH}"
done

printf "\nCreating SSDLC compliance report\n"
declare -r TEMPLATE_SSDLC_REPORT_PATH="${RELATIVE_DIR_PATH}/template_ssdlc_compliance_report.md"
declare -r SSDLC_REPORT_PATH="${SSDLC_PATH}/ssdlc_compliance_report.md"
cp "${TEMPLATE_SSDLC_REPORT_PATH}" "${SSDLC_REPORT_PATH}"
declare -a SED_EDIT_IN_PLACE_OPTION
if [[ "$OSTYPE" == "darwin"* ]]; then
SED_EDIT_IN_PLACE_OPTION=(-i '')
else
SED_EDIT_IN_PLACE_OPTION=(-i)
fi
sed "${SED_EDIT_IN_PLACE_OPTION[@]}" \
-e "s/\${product_name}/${PRODUCT_NAME}/g" \
-e "s/\${product_version}/${PRODUCT_VERSION}/g" \
-e "s/\${report_date_utc}/$(date -u +%Y-%m-%d)/g" \
"${SSDLC_REPORT_PATH}"
printf "%s\n" "${SSDLC_REPORT_PATH}"

printf "\n"
82 changes: 82 additions & 0 deletions .evergreen/template_ssdlc_compliance_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# ${product_name} SSDLC compliance report

This report is available at
<https://d-9067613a84.awsapps.com/start/#/console?account_id=857654397073&role_name=Drivers.User&destination=https%3a%2f%2fus-west-1.console.aws.amazon.com%2fs3%2fobject%2fjava-driver-release-assets%3fregion%3dus-west-1%26bucketType%3dgeneral%26prefix%3d${product_name}%2f${product_version}%2fssdlc_compliance_report.md>.

<table>
<tr>
<th>Product name</th>
<td><a href="https://github.com/mongodb/mongo-java-driver">${product_name}</a></td>
</tr>
<tr>
<th>Product version</th>
<td>${product_version}</td>
</tr>
<tr>
<th>Report date, UTC</th>
<td>${report_date_utc}</td>
</tr>
</table>

## Release creator

This information is available in multiple ways:

<table>
<tr>
<th>Evergreen</th>
<td>
Go to
<a href="https://evergreen.mongodb.com/waterfall/mongo-java-driver?bv_filter=Publish%20Release">
https://evergreen.mongodb.com/waterfall/mongo-java-driver?bv_filter=Publish%20Release</a>,
find the build triggered from Git tag <code>r${product_version}</code>, see who authored it.
</td>
</tr>
<tr>
<th>Papertrail, human-readable</th>
<td>
Go to
<a href="https://papertrail.devprod-infra.prod.corp.mongodb.com/product-version?product=${product_name}&version=${product_version}">
https://papertrail.devprod-infra.prod.corp.mongodb.com/product-version?product=${product_name}&version=${product_version}</a>,
look at the value in the "Submitter" column.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dylrich are we comfortable with this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to include these links in the report, maybe we should pull their templates from Evergreen project properties instead of checking them in to the repo.

Copy link
Contributor

@dylrich dylrich May 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The UI has no long-term stability guarantees and is intended purely for debugging. It's not an Official(tm) product we support. We have not received direction on an official way to serve Papertrail data yet. I'd be more comfortable with language that says something more vague like "see Papertrail"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 98c5113.

</td>
</tr>
<tr>
<th>Papertrail, JSON</th>
<td>
Go to
<a href="https://papertrail.devprod-infra.prod.corp.mongodb.com/product-version?product=${product_name}&version=${product_version}&format=json">
https://papertrail.devprod-infra.prod.corp.mongodb.com/product-version?product=${product_name}&version=${product_version}&format=json</a>
and look at the value associated with the <code>submitter</code> key.
</td>
</tr>
</table>

## Process document

Blocked on <https://jira.mongodb.org/browse/JAVA-5429>.

The MongoDB SSDLC policy is available at
<https://docs.google.com/document/d/1u0m4Kj2Ny30zU74KoEFCN4L6D_FbEYCaJ3CQdCYXTMc>.

## Third-darty dependency information

There are no dependencies to report vulnerabilities of.
Our [SBOM](https://docs.devprod.prod.corp.mongodb.com/mms/python/src/sbom/silkbomb/docs/CYCLONEDX/) lite
is <https://github.com/mongodb/mongo-java-driver/blob/r${product_version}/sbom.json>.

## Static analysis findings

The static analysis findings are all available at
<https://d-9067613a84.awsapps.com/start/#/console?account_id=857654397073&role_name=Drivers.User&destination=https%3a%2f%2fus-west-1.console.aws.amazon.com%2fs3%2fbuckets%2fjava-driver-release-assets%3fregion%3dus-west-1%26bucketType%3dgeneral%26prefix%3d${product_name}%2f${product_version}%2fstatic-analysis-reports%2f>.
All the findings in the aforementioned reports
are either of the MongoDB status "False Positive" or "No Fix Needed",
because code that has any other findings cannot technically get into the product.

<https://github.com/mongodb/mongo-java-driver/blob/r${product_version}/config/spotbugs/exclude.xml> may also be of interest.

## Signature information

The product artifacts are signed.
The signatures can be verified by following instructions at
<https://github.com/mongodb/mongo-java-driver/releases/tag/r${product_version}>.