Merge branch 'main' into rstuf-bootstrap

Author: Kairo Araujo

Makefile

@@ -74,7 +74,7 @@ static_pipeline: .state/docker-build-static
 reformat: .state/docker-build-base
 	docker compose run --rm base bin/reformat
 
-lint: .state/docker-build-base
+lint: .state/docker-build-base .state/docker-build-static
 	docker compose run --rm base bin/lint
 	docker compose run --rm static bin/static_lint
 

bin/rtd-docs

@@ -18,7 +18,7 @@ if [ "${READTHEDOCS_PROJECT}" = "docspypiorg" ]; then
   mkdir _readthedocs && mv docs/user-site _readthedocs/html
 fi
 
-if [ "${READTHEDOCS_PROJECT}" = "blogpypiorg" ]; then
+if [ "${READTHEDOCS_PROJECT}" = "blogpypiorg" ] || [ "${READTHEDOCS_PROJECT}" = "psf-warehouse-private" ]; then
   pip install -r requirements/docs-blog.txt
   asdf reshim
   mkdocs build -f docs/mkdocs-blog.yml

docker-compose.yml

@@ -148,7 +148,7 @@ services:
   worker:
     image: warehouse:docker-compose
     pull_policy: never
-    command: hupper -m celery -A warehouse worker -B -S redbeat.RedBeatScheduler -l info
+    command: hupper -m celery -A warehouse worker --beat --scheduler redbeat.RedBeatScheduler -l info
     volumes:
       - ./warehouse:/opt/warehouse/src/warehouse:z
       - packages:/var/opt/warehouse/packages

docs/blog/posts/2024-04-03-user-account-access.md

@@ -0,0 +1,146 @@
+---
+title: "Incident Report: Unauthorized User Accounts Access"
+description: An attack on PyPI user accounts starting on March 31st, 2024.
+authors:
+  - miketheman
+date: 2024-04-03
+tags:
+  - 2fa
+  - security
+  - transparency
+---
+
+On Sunday, March 31st, 2024, PyPI Admins received emails
+about unexpected account activity from PyPI users.
+Users received notifications from PyPI that they had
+enrolled in two-factor authentication (2FA).
+These users claimed that they had not done so themselves.
+
+PyPI Admins have **not found any evidence** of existing package tampering,
+or any other malicious activity beyond unauthorized account access and modification.
+
+The main actions post-investigation taken were:
+
+* affected accounts were frozen for further investigation
+* email re-verification was required for all accounts not yet enabled in 2FA
+
+Read on for a summary of what happened, how we responded, and what's next.
+
+<!-- more -->
+
+_All times are in Eastern Time (UTC-4), and some are approximated for clarity._
+
+## What Happened
+
+### Sunday, March 31st, 2024
+
+It was a weekend morning.
+
+* 06:20: PyPI Admins receive an influx of emails from users.
+
+These users were responding to an automated notification from PyPI
+that they had enrolled in two-factor authentication (2FA).
+
+**As a reminder, [PyPI requires all users](./2024-01-01-2fa-enforced.md) to enable 2FA.**
+
+Users stated they had not enrolled in 2FA themselves.
+The accounts in question had not previously enrolled in any form of 2FA,
+and the users were surprised by these notifications.
+
+Emails of this nature trickled in over the course of the day,
+as users read their email and notice the activity.
+
+* 08:00:  A volunteer PyPI Admin saw these notifications and began to investigate.
+
+They found some traffic patterns used for the attack.
+The admin then reached out to the rest of the team to discuss their findings,
+and to take action on the accounts that were affected.
+
+The main action taken was to immediately freeze any suspicious accounts,
+as this is completely reversible and would prevent any further damage
+to the known compromised accounts or projects they have access to.
+
+~120 user accounts were frozen due to the analysis of the situation,
+based on their characteristics and the traffic patterns observed.
+_(A precise count was not recorded as actions were taken in real-time.)_
+
+PyPI Admins began work on a feature which would require re-verification
+of email addresses for all accounts not yet enabled in 2FA,
+before any other account action can be taken (including enabling 2FA).
+
+* 11:00: the volunteer admin had to step away from the computer for the remainder of the day.
+
+Investigation continues on Monday.
+
+### Monday, April 1st, 2024
+
+Monday morning, investigations continue.
+
+* 08:00: an additional ~54 accounts were frozen based on similar criteria, with better filters
+
+Another investigation to determine if any packages had been tampered with began.
+**No evidence of package tampering was found** associated with any of these accounts.
+
+PyPI Admins completed the new feature to now require email address
+re-verification for any account that has not yet enabled 2FA.
+
+* 12:20: Account re-verification change went live.
+
+Once the change was in place, PyPI Admins took another action -
+invalidated any existing email verifications for accounts **without 2FA enabled**.
+This was done to prevent any unauthorized access to accounts that may have been compromised.
+Now, if a user who has not enrolled in 2FA logs in,
+they will be required to re-verify their email address,
+prior to enrollment in 2FA.
+This process ensures that someone logging into a pre-existing account
+without 2FA controls the password _and_ email address associated with the account.
+
+* 12:30: Email invalidation applied to 370k (56%) email addresses.
+
+The rationale was that these accounts were at risk of unauthorized access,
+and this was a proactive measure to prevent further damage.
+
+## Summary
+
+This event has similar characteristics to a
+[credential stuffing](https://attack.mitre.org/techniques/T1110/004/) attack,
+where attackers use credentials found via other service's leaks or incidents
+to attempt to log in to accounts on PyPI.
+
+For the time frame in question we did not observe a higher-than-normal
+rate of requests to the PyPI login endpoint.
+Authentication rate limits did not show any out-of-the-ordinary activity.
+
+This an attack that 2FA is designed to protect against,
+as even if the attackers have the correct username and password,
+they would still need to provide a second factor to gain access to the account.
+
+PyPI disallows the use of compromised passwords,
+commonly used in credential stuffing attacks,
+by [integrating](https://github.com/pypi/warehouse/blob/d16fc21b0d5a57409740f5ebd780ea446df6d95c/warehouse/accounts/services.py#L769-L825)
+with the [Have I Been Pwned](https://haveibeenpwned.com/) [API](https://haveibeenpwned.com/API/v3)
+to check for compromised passwords on login, but there was no evidence
+that these passwords were compromised in known data leaks on that platform
+at the time of login.
+
+All PyPI users should take precautions to secure their PyPI account
+and associated projects, including, but not only:
+
+* Use a unique password on PyPI
+* Verify your email address, add a secondary email address (optional)
+* Enable 2FA on your account. **Save your recovery codes somewhere safe!**
+* Review your packages for any unauthorized changes
+* Review collaborators on your projects
+
+For any users who have been affected by this incident,
+and cannot obtain access to their accounts with email re-verification,
+Please read the [FAQ on account recovery](https://pypi.org/help/#account-recovery).
+
+We will continue to monitor the situation and take further action as needed.
+
+---
+
+_This analysis, and my role, is made possible with the support from
+[Amazon Web Services Open Source](https://aws.amazon.com/opensource/),
+[Georgetown CSET](https://cset.georgetown.edu/),
+the [PSF](https://www.python.org/psf/) and their sponsors._

docs/dev/Makefile

@@ -175,3 +175,7 @@ pseudoxml:
 	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
 	@echo
 	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+
+# https://github.com/sphinx-doc/sphinx-autobuild#using-with-makefile
+livehtml:
+	sphinx-autobuild "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/dev/development/frontend.rst

@@ -8,14 +8,35 @@ serving.
 All of the static files are located in ``warehouse/static/`` and external
 libraries are found in ``package.json``.
 
+The static files are compiled and included in the
+``warehouse:docker-compose-static`` Docker image.
+
 
 Building
 --------
 
-Static files should be automatically built when ``make serve`` is running;
-however, you can trigger a manual build of them by installing
-`NodeJS 16.x <https://nodejs.org/en/download/releases/>`_, installing
-the dependencies using ``npm install`` and then running ``npm run build``.
+Static files should be automatically built when ``make serve`` is
+running; however, you can also manually run commands in the ``static``
+container:
+
+.. code-block:: console
+
+    $ # install dependencies
+    $ docker compose run --rm static npm install
+
+    $ # start a build
+    $ docker compose run --rm static npm run build
+
+
+Building outside of Docker
+--------------------------
+
+    Note: building outside of Docker is **not recommended** as it may
+    install platform-specific dependencies.
+
+Install `NodeJS 20.x <https://nodejs.org/en/download/releases/>`_,
+install the dependencies using ``npm install`` and then run ``npm run
+build``.
 
 If you're in a POSIX environment you may find
 `NVM <https://github.com/nvm-sh/nvm>`_ useful to have multiple NodeJS

package-lock.json

@@ -37,6 +37,7 @@
     "@babel/core": "^7.21.3",
     "@babel/eslint-parser": "^7.21.3",
     "@babel/preset-env": "^7.21.4",
+    "@stylistic/eslint-plugin-js": "^1.7.0",
     "@testing-library/dom": "^9.2.0",
     "@testing-library/jest-dom": "^5.16.5",
     "babel-jest": "^29.5.0",
@@ -77,24 +78,27 @@
     "parserOptions": {
       "sourceType": "module"
     },
+    "plugins": [
+      "@stylistic/js"
+    ],
     "rules": {
-      "comma-dangle": [
+      "@stylistic/js/comma-dangle": [
         "error",
         "always-multiline"
       ],
-      "indent": [
+      "@stylistic/js/indent": [
         "error",
         2
       ],
-      "linebreak-style": [
+      "@stylistic/js/linebreak-style": [
         "error",
         "unix"
       ],
-      "quotes": [
+      "@stylistic/js/quotes": [
         "error",
         "double"
       ],
-      "semi": [
+      "@stylistic/js/semi": [
         "error",
         "always"
       ]