This repository was archived by the owner on Nov 19, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Added Static and SemVer tests docs #7951
Merged
Merged
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
9a87b57
Added static and semver tests docs
ffaa20b
Renamed folder to svc
e3a05e3
Reverted undesired changes
a3179dc
Fixed issue with link to unit file
df21a73
Updated changes to avoid change on .dist files
3f948b3
Apply suggestions from code review
gabrieldagama f0d5e94
Replaced files with symlinks
bae328e
Fixed trailing spaces
785ebfa
Implemented suggestions
9af9830
Grammar and formatting
dobooth e106dc5
Grammar and formatting
dobooth c5565a6
Update src/guides/v2.3/test/svc/semver_test_execution.md
gabrieldagama b7cc4b3
Merge branch 'master' into updating-testing-docs
dobooth 4b692ff
Lint?
dobooth bb4d22b
Strange linting error
dobooth dbbe394
Linty
dobooth 0cf7de5
Merge branch 'master' into updating-testing-docs
dobooth b9b2816
Merge branch 'master' into updating-testing-docs
dobooth File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
--- | ||
group: testing | ||
title: Running Static Tests | ||
functional_areas: | ||
- Testing | ||
- test | ||
--- | ||
|
||
Executing Magento 2 static tests is straightforward. They can be executed in several ways. | ||
|
||
### Running static tests on all files | ||
|
||
To run static tests on all files, navigate to the Magento base directory and execute the following command: | ||
|
||
```bash | ||
bin/magento dev:test:run static | ||
``` | ||
|
||
### Running PHP static tests on a subset of files | ||
|
||
To run the static tests on a subset of files, create a new testsuite for phpunit: | ||
|
||
1. From the Magento base directory navigate to `dev/tests/static/testsuite/Magento/Test` | ||
|
||
1. Create a copy of the `Php` folder on the same directory and rename it to `Local` | ||
|
||
1. Navigate to `dev/tests/static/testsuite/Magento/Test/Local/_files/whitelist` and open `common.txt` | ||
|
||
1. Replace the contents with the folder of the files that you want to test. For example: | ||
|
||
```text | ||
# Format: <componentType=module|library|theme|language|*> <componentName> <globPattern> or simply <globPattern> | ||
app/code/Magento/CatalogSearch/Model/Search | ||
``` | ||
|
||
1. Create a copy of the `dev/tests/static/phpunit.xml.dist` file and call it `phpunit.xml`: | ||
|
||
1. Add a new testsuite to the `dev/tests/static/phpunit.xml` file inside the `<testsuites>` node: | ||
|
||
```xml | ||
<testsuites> | ||
... | ||
<testsuite name="Local Test Suite"> | ||
<file>testsuite/Magento/Test/Local/LiveCodeTest.php</file> | ||
</testsuite> | ||
</testsuites> | ||
``` | ||
|
||
1. Navigate to the Magento base directory and run: | ||
|
||
```bash | ||
./vendor/bin/phpunit --testsuite="Local Test Suite" -c dev/tests/static/phpunit.xml.dist | ||
``` | ||
|
||
As a result of this process, you can run PHP static tests on a subset of files. It is also possible to run other types of static tests by following the same process with other testsuites. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
--- | ||
group: testing | ||
title: Running Semantic Version Checker | ||
functional_areas: | ||
- Testing | ||
- test | ||
--- | ||
|
||
Executing the Magento 2 Semantic Version Checker is a simple process, but it requires an external tool. | ||
|
||
### Running Semantic Version Checker on local changes | ||
|
||
To run the Semantic Version Checker on local changes: | ||
|
||
1. Clone the `magento-semver` tool to your local machine: | ||
|
||
```bash | ||
git clone [email protected]:magento/magento-semver.git | ||
``` | ||
|
||
1. Navigate to the cloned repository and run `composer install` to install the project dependencies: | ||
|
||
```bash | ||
cd magento-semver && composer install | ||
``` | ||
|
||
1. To run the Semantic Version Checker, we need one folder with the feature changes and another folder with mainline code (without changes), so you may need to clone the Magento repository to a separate folder to perform the comparison. | ||
|
||
1. Navigate the `magento-semver` folder and run the Semantic Version Checker compare command: | ||
|
||
```bash | ||
bin/svc compare ../magento2-mainline ../magento2 | ||
``` | ||
|
||
The first parameter is the mainline code without any changes, and the second parameter is the path to the folder with your changes. | ||
The results of the Semantic Version Checker are outputted to the console. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../v2.3/test/static/static_test_execution.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../v2.3/test/svc/semver_test_execution.md |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrieldagama: out of curiosity, are these the exact same tests that are being ran when you submit a PR on github?
Unless something has changed recently, that's not the case. It seems like there are a bunch of extra static tests which are being executed on github.
I'm looking for easy to comprehend information about how to execute all these static tests on your local machine, it will save so much waiting time when working on a PR if we have this information.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @hostep, this is a valid point, thanks for bringing it up! Yes, they are the same tests, but they run a little bit differently, on our automated tests, the Static Tests are split in their separate test suite respectively (Less, HTML, PHP, Integrity...) and when you run locally, it runs all the suites at once. We should get the same results locally.
But there are somethings that I've noticed:
Maybe we can add this information to the docs as well, what do you think?
Hope this clarifies a little bit your question. If you have some PR that we can look at to see the differences it would be great!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok thanks for the info!
Ah yes, I seem to remember that it was almost always some extra tests in B2B that failed. Unfortunately we are only using Open Source so I have no access to those extra tests.
What is the reason for the static tests to be different on EE & B2B projects btw? These are static tests, in my opinion they should be 100% the same between the different type of Magento projects. I understand there is a difference for integration & functional tests, but I don't understand why there is a difference for static tests?
Would be great if the differences in static tests could get aligned so they are the same over all types of Magento projects, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with you, from what I can see the PHP check are exactly the same, it just changes some files that are ignored or not. But, there are some differences in other static tests, like Integrity and Legacy tests. I'm not fully aware of exactly the reasons and what are the differences, but I may be able to check internally.
But yeah, from my point of view, ideally we would have a defined test strategy that is publicly available and applied to all versions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also seem to remember vaguely that php mess detector static test ran on B2B and not on OS, but I might be mistaken