Description
Preconditions (*)
I'm addressing the issue that wasn't solved for more than 3 months already and is blocker for most of the contributions. In most of the cases the results of Functional Tests are ignored, but it wasn't the goal of running them (!)
- Magento is running multiple instances of tests on a single instance of the Store.
- There are multiple Test Suites - eg.
SearchEngineElasticsearchSuitethat expect indices to workOn schedulemode. - Data is created using
<createDatavia API - A tested instance of Magento is not running Cronjobs (!)
Steps to reproduce (*)
- Install Magento instance locally
- Set indices to work
On schedule - Execute any test that
<createData
Expected result (*)
- We expect the test to pass, of course
Actual result (*)
- But the test will probably fail, as the Cronjob was not run so the partial reindex was not executed.
Solution 🎉
- Add extra command after each block of
<createDatas like:<magentoCron stepKey="runCronIndex" groups="index"/>to make sure that Cronjobs ran the partial reindex on specific items (products, categories).
Controverses
There are people who deeply believe that running Magento cronjobs is "hiding the issue" and they prefer to run index:reindex and cache:flush. This is unresponsible behaviour and should not be applied! The same people believe that cron:run executed twice one after another makes a difference. (Try it yourself: First will be executed, second one skipped if it's run in less than 60 seconds)
Problems
I still haven't managed why running Cronjobs are not reindexing the links in Configurable / Grouped products, but even if those products will still cause an issue, most of the Test cases are going to pass.
Modules affected
- Magento_Analytics (Tests)
- Magento_Backend (Tests)
- Magento_Braintree (Tests)
- Magento_Bundle (ActionGroups)
- Magento_Bundle (Tests)
- Magento_Captcha (Tests)
- Magento_Catalog (Tests)
- Magento_CatalogImportExport (Tests)
- Magento_CatalogInventory (Tests)
- Magento_CatalogRule (Tests)
- Magento_CatalogRuleConfigurable (Tests)
- Magento_CatalogSearch (Tests)
- Magento_CatalogUrlRewrite (Tests)
- Magento_CatalogWidget (Tests)
- Magento_Checkout (Tests)
- Magento_CheckoutAgreements (Tests)
- Magento_Cms (Tests)
- Magento_Config (ActionGroups)
- Magento_Config (Tests)
- Magento_ConfigurableProduct (ActionGroups)
- Magento_ConfigurableProduct (Tests)
- Magento_CurrencySymbol (Tests)
- Magento_Customer (Tests)
- Magento_Downloadable (Tests)
- Magento_Elasticsearch (Tests)
- Magento_Elasticsearch6 (Tests)
- Magento_Fedex (Tests)
- Magento_GroupedProduct (Tests)
- Magento_ImportExport (Tests)
- Magento_LayeredNavigation (Tests)
- Magento_Msrp (Tests)
- Magento_Multishipping (Tests)
- Magento_Newsletter (Tests)
- Magento_OfflineShipping (Tests)
- Magento_PageCache (Tests)
- Magento_Paypal (Tests)
- Magento_Persistent (Tests)
- Magento_ProductVideo (Tests)
- Magento_Quote (Tests)
- Magento_Reports (Tests)
- Magento_Review (Tests)
- Magento_Sales (Tests)
- Magento_SalesRule (Tests)
- Magento_Search (Tests)
- Magento_Security (Tests)
- Magento_Shipping (ActionGroups)
- Magento_Shipping (Tests)
- Magento_Store (Tests)
- Magento_Swatches (Tests)
- Magento_Tax (Tests)
- Magento_Translation (Tests)
- Magento_Ui (Tests)
- Magento_Ups (Tests)
- Magento_UrlRewrite (Tests)
- Magento_User (ActionGroups)
- Magento_User (Tests)
- Magento_Vault (Tests)
- Magento_Weee (Tests)
- Magento_Widget (Tests)
- Magento_Wishlist (Tests)
(This is my checklist to confirm that task was done - please don't touch!)