Cannot save changes in large categories

[Schoeny1]

Hello,

Precondtions

Using Magento 2.1.3, PHP 5.6, MySQL 5.6

I have a category with over 10,000 products. I cannot seem to get the display mode to save.

Steps to reproduce

In the admin

1. Choose category
2. Change Display mode to Static Blocks Only
3. Save

Expected result

1. Category should save successfully

Actual result

I first was getting PHP timeouts:
increased it all the way to 30 mins and now I ran out of memory for the PHP script.
Is this normal for such a large category?
Are there limitations or recommendations for category size?

[pboisvert]

What is the business purpose to having a category with so many products? This will be poor customer experience from both a page load time and usefulness of the category page itself for browsing.

[maghamed]

For now Magento has a limitation of 10000 products assigned to one category.
This limitation was chosen taking into account that category which has huge amount of products will worsen customer experience and will lead to performance degradation of the whole store. Thus providing ability to create such a big category will introduce potential vulnerability for Magento store. Because the same limitation is used for Site Search functionality (actually Category Page is Site Search as well, because Category View uses Site search under the hood). The more products returned from Search -> the slower page generation time would be. Search Result pages are not cached with Full Page Cache. So, hacker could generate lot of Search queries to Magento store (which return more than 10000+ products) what will make Magento store stuck with performance problem.

Magento has also configuration value devoted for how much products should be displayed on Search/Advanced Search/Category page

It could be found in CatalogSearch/etc/search_request.xml
<size>10000</size>

[Schoeny1]

Thank you for the feedback, that is exactly what I needed to know. I am helping develop the site and I don't really have any control over the amount of products in the category. But now I have a reason to go back to the client and explain why this is a bad practice.

[JstrzAtW2M]

We have clients that pull in products from a third party and they don't receive categories. The categories are added manually and never shown on the site. There are filter dropdowns used on the home page to then find the products. Also, the pages have paging so they don't return all 10,000 or 5000 etc products all at once but only a subset. We've used this in Magento 1 and it worked.

We've noticed timeouts and issues with any category that has more then 2500 to 3000 products.
I've also tried saving parent categories that have subcategories totaling more than 4000 products and they also timeout. Oddly, the default category does save.

Since I've worked with other ecommerce platforms were this wasn't an issue, including Magento 1. I feel this is a problem and limitation that should be addressed

[maghamed]

@jstrez77
it doesn't matter whether you have pagination or not. Because products on the page are sorted (by Relevance, Name or Price). So you should get them all together and sort out. And just after that apply pagination.
And if you have Category Page which has more than 10000 products inside. You should build LN for each filterable attribute for all those products.

Regarding poor performance on big Category pages (2500, 3000, 4000 products). We don't have limitations which prevent you from having such categories. The reasons for timeout you experience could be very different.
Depending on answers:

• whether you have OPcode cache enabled?
• do you use NFS or other network file system like CIFS?
• what version of PHP do you use (5.6.* or 7.0.*)?
• how does php running on your server? PHP-FPM?
• do you use Swatches functionality?
• do you use any 3rd party module(-s)?
• how does I/O stats looks at your application server?
• do you have lot of images assigned to products in your catalog?

Use blackfire profiler to see where does the bottleneck come from.
So, most likely you have a problem with environment or configuration.

[magento-engcom-team]

@Schoeny1, @jstrez77 we are closing this issue due to inactivity. If you'd like to update it (according to the comment above), please reopen the issue.

[orlangur]

Like many other memory/time consumption issues related to saving large categories I believe this one is related to poor URL Rewrites performance.