MariaDB Casting is too slow

[osrecio]

When the MariaDB is try to cast the entitys_ids from string to integer is to slow,

Check: #27129

Description (*)

Related Pull Requests

Fixed Issues (if relevant)

1. Fixes magento/magento2#<issue_number>

Manual testing scenarios (*)

1. Change value for simple products to 50000 for example (setup/performance-toolkit/profiles/ce/small.xml);
2. Run in console: bin/magento setup:perf:generate-fixtures /var/www/html/magento24/setup/performance-toolkit/profiles/ce/small.xml for example
3. Reindex

In this PR, when you try to reindex EAV, the query can be vary from:

• [before] with quotes -> 7882 rows in set (34.460 sec)
• [after] no quotes -> 7882 rows in set (0.086 sec)

Questions or comments

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

Resolved issues:

1. resolves [Issue] MariaDB Casting is too slow #35568: MariaDB Casting is too slow

[m2-assistant]

Hi @osrecio. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE,
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

For more details, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[ihor-sviziev]

Hi @osrecio,
Your changes look good to me.
Could you please provide steps to reproduce the issue?

[osrecio]

Hi @ihor-sviziev ,

I added some manual steps, seems like in the previous PR, not all possibilities were covered.

In this PR, when you try to reindex EAV, the query can be vary from:

• [before] with quotes -> 7882 rows in set (34.460 sec)
• [after] no quotes -> 7882 rows in set (0.086 sec)

[ihor-sviziev]

@osrecio thank you!

@magento run all tests

PS: Added label of "auto-tests: not required" because such case would be really hard to test automatically + it's reproducing the only on some specific environments (MariaDB should be used as a DB)

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[ihor-sviziev]

@magento create issue

[ihor-sviziev]

✔ Approved.

Failing tests look not related to changes from this PR.

[engcom-Alfa]

@magento Functional Tests B2B, Functional Tests CE, Functional Tests EE

[engcom-Alfa]

Hi @osrecio & @ihor-sviziev Thanks for your contribution and collaboration!
I tried to test the performance improvements with the fixes and below is my observation

1. Made sure that, I have products with the count 50k+, that I got using performance toolkit file large.xml.
2. From the Magento root directory, executed the command: php bin/magento indexer:reindex catalog_product_attribute
3. Noted the time taken and it was Product EAV index has been rebuilt successfully in 00:08:32
4. Later, I pulled this PR file changes into my current Magento instance, and re-ran the same command.
5. Observed that, the tike taken is Product EAV index has been rebuilt successfully in 00:08:34. Below is the screenshot of prior code changes and after code changes output.

Here, noticeable thing is, it has taken 2 seconds more time to the prior file changes execution. I re-tried the same scenario twice in my system and found the same 2 seconds more time taken both the times.
Can you please check once and let me know if I have missed anything here. Or whether may require any more code changes from your end too!

[engcom-Alfa]

Please instruct me in case of any more steps if I have to follow based on my above-comment, and move this ticket back to Ready For Testing.

[ihor-sviziev]

@engcom-Alfa did you check on MariaDB database?

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[ishakhsuvarov]

It would be nice to get an additional round of testing on this one. Changes look like they should not be casing any degradation in perofmance numbers to say the least.

Moving to extended testing

[engcom-Charlie]

@engcom-Lima as per this comment, please have a look at this.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Alfa]

@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Lima]

Hi @osrecio ,

Thanks for your contribution and collaboration.

As per your comment it should take very less time to reindex EAV.

In this PR, when you try to reindex EAV, the query can be vary from:

[before] with quotes -> 7882 rows in set (34.460 sec)
[after] no quotes -> 7882 rows in set (0.086 sec)

But I have observed It is still taking longer time to reindex EAV. Below are the steps which I have followed -

1. Installed magento 2.4-develop using mariadb.
2. Creating 5 lakh products using performance fixtures (large profile)
3. Ran bin/magento indexer:reindex catalog_product_attribute

Time taken before PR change:

Time taken after PR change:

From above screenshots we can see time difference is only 1 min. Can you provide information about it.

Thanks

[engcom-Lima]

Hi @osrecio ,

Please provide information for above comment so, that we can proceed further as soon as possible.

Thanks

[engcom-Lima]

@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Lima]

✔️ QA Passed

Preconditions:

• Install fresh Magento 2.4-develop

Manual testing scenario:

1. Installed magento 2.4-develop using mariadb.
2. Creating 5 lakh products using performance fixtures (large profile)
3. Ran bin/magento indexer:reindex catalog_product_attribute

Before: ✖️

After: ✔️ Observed time difference of 1 min.

No additional manual test case is required as part of regression cycle.