diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 6324a64fe7..bf9d977c1e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -4,7 +4,7 @@ name: New p5.js release on: push: tags: - - 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10 + - 'v*.*.*' # Push events to matching v*.*.*, i.e. v20.15.10 jobs: release: diff --git a/contributor_docs/release_process.md b/contributor_docs/release_process.md index bdf3a57745..1cf85f93c4 100644 --- a/contributor_docs/release_process.md +++ b/contributor_docs/release_process.md @@ -3,42 +3,52 @@ ## Approach * We follow the [semver](https://semver.org/) versioning pattern, which means we follow this versioning pattern: `MAJOR:MINOR:PATCH`. - ## Requirements -* Git, node.js (v10+) and NPM installed on your system -* Logged in NPM CLI : Check if you are logged in by `npm whoami` -* High Bandwidth : Lots of things to download/pull/push (\~190 MB total I presume) +* Git, node.js and NPM installed on your system +* You are able to build the library and have push access to the remote repository +* Secret `NPM_TOKEN` value is set on the remote repository +* Secret `ACCESS_TOKEN` value is set on the remote repository ## Usage -```shell -$ npm run release +```sh +$ git checkout main +$ npm version [major|minor|patch] # Choose the appropriate version tag +$ git push origin main +$ git push origin v1.4.2 # Replace the version number with the one just created above ``` +The actual release steps are all run on Github Actions CI. After the action has finished running, you may want to view the release on Github and modify the release note if required (eg. separate all-contributor bot commits with other commits). + +## Security tokens +For the release steps to run fully, two [repository secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) must be set as below. -* This will run the build steps and you should follow the prompt provided by `np` to finish the process. -* The build step then proceed to the grunt task of creating a zip version of the library as well as releasing on bower and releasing the reference on the website. +* `NPM_TOKEN` can be created by following the steps [here](https://docs.npmjs.com/creating-and-viewing-access-tokens) to create a read and publish token. The user the token belongs to must have publish access to the project on NPM. +* `ACCESS_TOKEN` is a personal access token for a user that has access to `p5.js`, `p5.js-website`, and `p5.js-release` repositories. The token can be generated using the steps [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) and for scopes, choose only `public_repo`. It is recommended to use an organization specific account for this (ie. not from a personal account) and limit the account's write access to only the required repositories. ## What's actually happening -* `npm run release` is an alias of `grunt release-p5` which will first spawn a child process running [`np`](https://www.npmjs.com/package/np). -* `np` will start by checking your local repository and settings are ready for you to create a release. You will need to have no uncommitted changes in the local repository in order to continue. -* `np` will then reinstall `node_modules` then run tests with `npm test`. -* `np` will bump the version according to what was selected at the beginning. -* If any step prior to this failed, the repo will be reverted to its initial state before running `npm run release`. -* The task mentioned in `prepublishOnly` in `package.json` will run ( `grunt prerelease` ) to build the documentation and the library with the updated version number -* The NPM package is published. - * Release on NPM : __Only__ the files mentioned in `files` in `package.json` are published. -* Tags and local commits are pushed to the git remote. -* A draft release is created on github.com with changelogs that can be edited. -* Create a Zip file `p5.zip` of `lib` folder (now includes the empty example), which should be uploaded in the GitHub Release draft created above. - * After this process completes a window pointing at `release/` will open and it will contain all the files that should be uploaded as part of the Github Release. -* Push the newly built library to [p5.js-release](https://github.com/processing/p5.js-release) repo for Bower. -* Push the newly built reference to [p5.js-website](https://github.com/processing/p5.js-website) +The Github Action ["New p5.js release"](../.github/workflows/release.yml) is triggered on a tag that matches the pattern `v*.*.*` which is created by the `npm version ___` command. + +Once triggered, it will run the following steps: + +1. Clone the repository, setup node.js, extract version number, install dependencies with `npm`, and run test with `npm test`. +2. Create the release files that will be uploaded to Github releases. +3. Create a release on Github and publish latest version on NPM. +4. Update website files + 1. Clone the website repository + 2. Copy `data.json` and `data.min.json` to the right location + 3. Copy `p5.min.js` and `p5.sound.min.js` to the right location + 4. Update `data.yml` file with latest version number + 5. Update `en.json` file based on `data.min.json` + 6. Commit and push the changes back to the website repository +5. Update Bower files + 1. Clone the Bower release repository + 2. Copy all libraries files to the right location + 3. Commit and push the changes back to the website repository + +In principle, we try to concentrate as many steps as possible to be run in one place, ie. in the CI environment. If a new step that is only run on release is required, it should probably be defined in the CI workflow and not as part of the build configuration. ## Testing -In the case where you have push access to the repositories: -* You can run `npm run release -- --preview` to do a dry run of the release process. No git tracked files will be changed by running this step and no push will be made to any of the remotes. +As the release steps are run in CI, testing them can be difficult. Using [act](https://github.com/nektos/act) to test running of the steps locally is possible (and was how they were tested while being developed) but require some temporary modifications to the workflow definition, we'll roughly document here as the precise steps will likely change over time. -In the case where you don't have push access to the repositories: -* You will need to edit the `name` field of `package.json` to a namespaced version, eg. `@username/p5` and commit this change into git before running `npm run release -- --preview` as usual. When prompted just choose not to publish the package to the namespaced packaged on NPM, nothing will be published online. -* You can do a full test run of the release with `npm run release` provided you have edited the `name` field of `package.json`. To choose where to clone and push the Bower release and website repositories from, you can set them by specifying additional arguments like so: `npm run release -- --bowerReleaser=username --docsReleaser=username`. +The test steps will not run because not all system requirements are present to run the mocha Chrome tests. Some system dependencies will likely be needed to be installed with `apt` before setting up the rest of the environment. Keep an eye on the error messages which should give some information on what packages are missing. -__NOTE:__ `np` (`6.2.0`) currently has a [bug](https://github.com/sindresorhus/np/issues/508) that prevents release to namespaced package name, you can revert to `5.2.1` if you must test this otherwise it will fail at the publish step. \ No newline at end of file +The steps concerning pushing changes to remote repositories should be commented out to avoid accidentally pushing unintended changes. \ No newline at end of file diff --git a/contributor_docs/steward_guidelines.md b/contributor_docs/steward_guidelines.md index d881323204..5f5e11f5e3 100644 --- a/contributor_docs/steward_guidelines.md +++ b/contributor_docs/steward_guidelines.md @@ -5,16 +5,21 @@ Whether you have just joined us as a steward, a seasoned maintainer of p5.js, or - [Issues](#issues) - [Bug report](#bug-report) - [Feature request](#feature-request) - - [Feature enhancement](#feature-enchancement) + - [Feature enhancement](#feature-enhancement) - [Discussion](#discussion) - [Pull Requests](#pull-requests) - [Simple fix](#simple-fix) - [Bug fix](#bug-fix) - - [New feature/feature enchancement](#new-feature-feature-enchancement) + - [New feature/feature enhancement](#new-feature-feature-enhancement) - [Dependabot](#dependabot) - [Build Process](#build-process) + - [Main build task](#main-build-task) + - [Miscellaneous tasks](#miscellaneous-tasks) - [Release Process](#release-process) - [Tips & Tricks](#tips--tricks) + - [Reply templates](#reply-templates) + - [Github CLI](#github-cli) + - [Managing notifications](#managing-notifications) --- @@ -64,16 +69,16 @@ For feature request issues, they should be using the "New Feature Request" issue - Will it conflict with typical sketches already written for p5.js? - Features that are likely to break the above should be considered breaking changes and without a major version release, we should not make breaking changes to p5.js. - Can the proposed new feature be achieved using existing functionalities already in p5.js, relatively simple native Javascript code, or existing easy to use libraries? - - Eg. instead of prividing a p5.js function to join an array of strings such as `join(["Hello", "world!"])`, the native Javascript `["Hello", "world!"].join()` should be preferred instead. + - Eg. instead of providing a p5.js function to join an array of strings such as `join(["Hello", "world!"])`, the native Javascript `["Hello", "world!"].join()` should be preferred instead. 3. Provided the access requirement and other considerations have been fulfilled, at least two stewards or maintainers must approve the new feature request before work should begin towards a PR. The PR review process for new features is documented below. -## Feature enchancement -For feature enchancement issues, they should be using the "Existing Feature Enhancement" issue template. The process here very similar to new feature request. The difference between new feature request and feature enchancement can be blurred however feature enchancement mainly deals with existing functions of p5.js while new feature request could be requesting entirely new functions to be added. +## Feature enhancement +For feature enhancement issues, they should be using the "Existing Feature Enhancement" issue template. The process here very similar to new feature request. The difference between new feature request and feature enhancement can be blurred however feature enhancement mainly deals with existing functions of p5.js while new feature request could be requesting entirely new functions to be added. -1. Similar to new feature request, feature enchancement should only be accepted if they increases access of p5.js. Please see point 1 of [section above](#feature-request) -2. Inclusion criterias for feature enchancement are similar to those for feature request above but particular attention should be paid to potential breaking changes. +1. Similar to new feature request, feature enhancement should only be accepted if they increases access of p5.js. Please see point 1 of [section above](#feature-request) +2. Inclusion criterias for feature enhancement are similar to those for feature request above but particular attention should be paid to potential breaking changes. - If modifying existing functions, all previous valid and documented function signatures must behave in the same way. -3. Feature enchancements must be approved by at least one steward or maintainer before work should begin towards a PR. The PR review process for feature enchancement is documented below. +3. Feature enhancements must be approved by at least one steward or maintainer before work should begin towards a PR. The PR review process for feature enhancement is documented below. ## Discussion This type of issue has a minimal template ("Discussion") and should be use only if a particular discussion doesn't fall under the other three existing templates or be better suited to the forum or Discord. @@ -91,7 +96,7 @@ Almost all code contribution to the p5.js repositories happens through pull requ - Almost all pull requests must have associated issues opened and discussed first, meaning the relevant [issue workflow](#issues) must have been followed first before a PR should be reviewed by any steward or maintainer. - The only instance where this does not apply are very minor typo fixes, which does not require an opened issue and can be merged by anyone with merge access to the repo, even if they are not stewards of a particular area. - While this exception exist, we will apply it in practice only but contributors are usually encouraged to open new issues. In other words, if in doubt about whether this exception applies, just open an issue anyway. -- If a pull request does not fully solve the referenced issue, you can edit the original post and change "Resovles #OOOO" to "Addresses #OOOO" so that it does not automatically close the original issue when this PR is merged. +- If a pull request does not fully solve the referenced issue, you can edit the original post and change "Resolves #OOOO" to "Addresses #OOOO" so that it does not automatically close the original issue when this PR is merged. ## Simple fix Simple fix such as small typo fix can be merged directly by anyone with merge access with a quick check on the PR "Files Changed" tab and that automated CI test passes. @@ -116,10 +121,10 @@ Simple fix such as small typo fix can be merged directly by anyone with merge ac @all-contributors please add @[github handle] for [contribution type] ``` -## New feature/feature enchancement +## New feature/feature enhancement The process for new feature or feature enhancement PR is similar to bug fixes with just one notable difference. -- A new feature/feature enchancement PR must be reviewed and approved by at least two stewards or maintainer before it can be merged. +- A new feature/feature enhancement PR must be reviewed and approved by at least two stewards or maintainer before it can be merged. - This can be the same two stewards or maintainer that approved the original issue or not. ## Dependabot @@ -133,11 +138,153 @@ Dependabot PRs are usually only visible to repo admins so if this does not apply --- # Build process +This section will not cover the general build setup nor commands but rather details about what's happening behind the scenes. For the above, please see the [contributors guidelines](./contributor_guidelines.md#working-on-p5js-codebase). + +The Gruntfile.js document contains the main build definitions for p5.js and more. Among the different tools used to build the library and documentation includes but not limited to Grunt, Browserify, YUIDoc, ESLint, Babel, Uglify, and Mocha. It may be helpful for us to start with the `default` task and work backwards from there. + +## Main build task + +```js +grunt.registerTask('default', ['lint', 'test']); +``` +When we run `grunt` or the npm script `npm test`, we run the default task consisting of `lint` then `test`. + +```js +grunt.registerTask('lint', ['lint:source', 'lint:samples']); +``` +The `lint` task consist of two sub tasks `lint:source` and `lint:samples`. `lint:source` is further subdivided into three more sub tasks `eslint:build`, `eslint:source`, and `eslint:test`, which uses ESLint to check the build scripts, the source code, and the test scripts. + +The `lint:samples` task will first run the `yui` task which itself consists of `yuidoc:prod`, `clean:reference`, and `minjson`, which extract the documentation from the source code into a JSON document, remove unused files from the previous step, and minify the generated JSON file into `data.min.json` respectively. Next in `lint:samples` is `eslint-samples:source` which is a custom written task whose definition is in [./tasks/build/eslint-samples.js](./tasks/build/eslint-samples.js), and it will run ESLint to check the documentation example code to make sure they follow the same coding convention as the rest of p5.js (`yui` is run first here because we need the JSON file to be built first before we can lint the examples). + +```js +grunt.registerTask('test', [ + 'build', + 'connect:server', + 'mochaChrome', + 'mochaTest', + 'nyc:report' +]); +``` +First let's look at the `build` task under `test`. +```js +grunt.registerTask('build', [ + 'browserify', + 'browserify:min', + 'uglify', + 'browserify:test' +]); +``` +Tasks that start with `browserify` are defined in [./tasks/build/browserify.js](./tasks/build/browserify.js). They are all essentially running the same steps with just minor differences. These are the main steps to build the full p5.js library from its many source code files into one, `browserify` builds p5.js while `browserify:min` builds an intermediate file to be minified in the next step. The difference between `browserify` and `browserify:min` is that `browserify:min` does not contain data needed for FES to function. `uglify` takes the output file of `browserify:min` and minify it into the final p5.min.js (configuration of this step is in the main Gruntfile.js). `browserify:test` is building a version identical to the full p5.js except for added code that is used for test code coverage reporting (using Istanbul). + +In the browserify steps, in addition to bundling the various files into one, a few more steps are also performed here. First, uses of `fs.readFileSync()` node.js specific code is replaced with the file's actual content using `brfs-babel`. This is used mainly by WebGL code to inline shader code while having them as separate files. + +Next, the source code including all dependencies from node_modules are transpiled using Babel to match the Browserslist requirement defined in package.json as well as to make the ES6 import statements into CommonJS `require()` that browserify understands. This also enables to use newer syntax available in ES6 and beyond without worrying about browser compatibility much. + +After bundling but before the bundled code is written to file, the code is passed through `pretty-fast`, if it is not meant to be minified, to be cleaned up so the final formatting is a bit more consistent (we anticipate the p5.js source code can be read and inspected if desired). + +A few small detailed steps are left out here, you can checkout the browserify build definition file linked above to have a closer look at everything. + +``` +connect:server +``` +This step spins up a local server hosting the test files and built source code files so that automated tests can be run in Chrome. + +``` +mochaChrome +``` +This step is defined in [./tasks/test/mocha-chrome.js](./tasks/test/mocha-chrome.js). It uses Puppeteer to spin up a headless version of Chrome that can be remote controlled and runs the tests associated with the HTML files in the `./test` folder, which includes testing the unminified and minified version of the library against the unit test suites as well as testing all reference examples. + +``` +mochaTest +``` +This step differs from `mochaChrome` in that it is run in node.js instead of in Chrome and only test a small subset of features in the library. Most features in p5.js will require a browser environment so this set of tests should only be expanded if the new tests really don't need a browser environment. + +``` +nyc:report +``` +Finally after all build and tests are complete, this step will gather the test coverage report while `mochaChrome` was testing the full version of the library and print the test coverage data to the console. Test coverage for p5.js is mainly for monitoring and having some additional data points, having 100% test coverage is not a goal for us. + +And that covers the default task in the Gruntfile.js configuration! + +## Miscellaneous tasks +All of the steps, sub-steps, sub-sub-steps can all be run directly with `npx grunt [step]` if wished to although for some it may not make much sense to do so when it depends on the earlier steps in this chain. There are also a few tasks that are not covered above but could be useful in certain cases. + +``` +grunt yui:dev +``` +This task will run the documentation and library builds described above followed by spinning up a web server that serves a functionally similar version of the reference page you will find on the website on [http://localhost:9001/docs/reference/](http://localhost:9001/docs/reference/). It will then monitor the source code for changes and rebuild the documentation and library. + +This is useful when you are working on the reference in the inline documentation because you don't have to move built files from the p5.js repository to a local p5.js-website repository and rebuild the website each time you make a change, and you can just preview your changes with this slightly simplified version of the reference in your browser. This way you can also be more confident that changes you made are likely to show up correctly on the website. Note that this is only meant for modifications to the inline documentation, changes to the reference page itself including styling and layout should be made and testing on the website repository. + +``` +grunt watch +grunt watch:main +grunt watch:quick +``` +The watch tasks will watch a series of files for changes and run associated tasks to build the reference or the library according to what files have changed. These tasks all do the same thing with the only difference being the scope. + +The `watch` task will run all builds and tests similar to running the full default task on detecting changes in the source code. + +The `watch:main` task will run the library build and tests but not rebuild the reference on detecting changes in the soruce code. + +The `watch:quick` task will run the library build only on detecting changes in the source code. + +Depending on what you are working on, choosing the most minimal watch task here can save you having the manually run a rebuild whenever you want to make some changes. --- # Release process +Please see [release_process.md](./release_process.md). --- -# Tips & tricks \ No newline at end of file +# Tips & tricks +Sometimes the amount of issues and PR that require review can get a bit overwhelming, while we try to put in place processes that make things easier, there are some tips and tricks that you can utilize to help with reviewing issues and PRs. + +## Reply templates +A handy GitHub feature that you can use is the [Saved Replies](https://docs.github.com/en/get-started/writing-on-github/working-with-saved-replies/about-saved-replies) feature available to use when authoring a reply to issues or pull requests. Some of the workflow describe above may require responding to issues or PRs with identical or very similar replies (redirecting question to forum, accepting an issue for fixing, etc) and using Saved Replies can just ever so slightly make this more efficient. + +Below are some of the Saved Replies that are being used by p5.js maintainers, you can use them for yourself or create your own! + +#### Closing: Can’t Reproduce +> We're not able to reproduce this but please feel free to reopen if you can provide a code sample that demonstrates the issue. Thanks! + +#### Closing: Need Snippet +> I'm closing this for organizational purposes. Please reopen if you can provide a code snippet that illustrates the issue. Thanks! + +#### Closing: Use the Forum! +> The github issues here are a good place for bugs and issues with the p5.js library itself. for questions about writing your own code, tests, or following tutorials, the [forum](https://discourse.processing.org/) is the best place to post. Thanks! + +#### Closing: GSOC +> Thanks! The best place to discuss GSOC proposals is on our [forum](https://discourse.processing.org/c/summer-of-code). + +#### Closing: Access +> I'm not seeing a lot of interest in this feature, and we don't have a clear explanation of how it [expands access](https://github.com/processing/p5.js/blob/main/contributor_docs/access.md), so I will close this for now. If an access statement can be added to the issue request, please feel welcome to reopen. + +> We are not seeing a further explanation of how this issue [expands access](https://github.com/processing/p5.js/blob/main/contributor_docs/access.md), so I will close this issue for now. If a more detailed access statement can be added to the feature request, please feel welcome to reopen it. Thank you! + +#### Closing: Addon +> I think this function is beyond the scope of the p5.js API (we try to keep as minimal as possible) but it could be a great starting point for an addon library. see the docs here for how to create an addon: https://github.com/processing/p5.js/blob/main/contributor_docs/creating_libraries.md + +#### Closing PR: Need Issue First +> Thank you. As a reminder, issues need to be opened before pull requests are opened and tagged with the issue. This is necessary for tracking development and keeping discussion clear. Thanks! + +#### Approve issue for fixing +You can go ahead with a fix. Thanks. + +#### Merged PR +Looks good. Thanks! + +## Github CLI +Reviewing a complex PR can be difficult with seemingly arcane git commands required to get the PR's version of code locally for you to test. Fortunately the [GitHub CLI](https://cli.github.com/) tool can help greatly with this process and more. + +After installing the CLI and logging in, reviewing a PR locally can be done by just running the command `gh pr checkout [pull_request_id]` and the process of fetching remote fork, creating a branch, and checking out the branch are all done automatically for you. Going back to the main branch will just be the same as switching a branch by running `git checkout main`. You can even leave a comment in the PR from the CLI without needing to visit the webpage at all! + +There are many other commands available in the GitHub CLI as well that you may or may not find useful, but it is a good tool to have around in any case. + +## Managing notifications +Instead of manually monitoring the "Issues" or "Pull Requests" tabs of the repo for new issues or PRs, you can "watch" the repo by clicking on the "Watch" button with an eye icon on the top of the repo page opposite the repo name. By watching a repo, events such as new issues, new pull requests, mentions of your user handle, and other activities you subscribed to on the repo will be sent as notification to your [notification page](https://github.com/notifications) where they can be marked as read or dismissed much like an email inbox. + +In some cases you may received emails from GitHub about events in repo you are watching as well and you can customize these (including unsubcribe from them completely) from your [notifications settings page](https://github.com/settings/notifications). + +Setting these up to fit the way you work can be the difference between having to find relevant issues/PRs to review manually and being overwhelmed by endless notifications from GitHub. A good balance is required here. As a starting suggestion, stewards should watch this repo for "Issues" and "Pull Requests" and set to only receive email on "Participating, @mentions and custom". \ No newline at end of file