Magento 2.4.1 - HUGE Cache Sizes grows quickly

[amitvkhajanchi]

Magento CE 2.4.1
Ubuntu 20.04 LTS
PHP 7.4.3
MySQL 8.022 (Amazon RDS)
Elasticsearch 7.6.2
Redis 5.2.1
Varnish 6.2.1
Nginx 1.8

PS. I have over 5000+ products in 2 websites (EU/US) EU site has 5 Store views (EN,FR,IT,DE,ES). US site has 1 Store view (US). Total of 6 Store view for my site.

During Testing, the site was performing well on my staging server. When I finally deployed it to production this week after setting the mode to production and seeing real traffic, my cache started growing at alarming rate. Right now, it will fill up 100GB in 2 hours. That never happened in Magento 2.3.3 in my old box.

My old box had 30GB in production mode and was running fine with Varnish. much smaller.
When I run the command du -h /var/cache it total 741MB in my old box
When I run the command du -h /var/cache in my current box it grows from 0 -> 10GB in 1 min and within 2 hours its 100GB.

Attached are results of both du command to see file size.
du-output-2.4.1-mage--d.txt
du-output-2.3.3-mage--d.txt

[m2-assistant]

Hi @amitvkhajanchi. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[hostep]

Since your block caches are very big, I believe this might be a duplicate of #29964 which was fixed in Magento 2.4.2

You can try to temporarily disable the Magento_Csp module and see if that solves it. If that is the case you'll be certain that it's the same bug and upgrading to 2.4.2 should fix it.

[amitvkhajanchi]

@hostep

Thanks for quick response. I have disabled the module. I am using Amazon EFS for the pub/media folder and realized that I had encryption on for this. I created another unencrypted volume and mounted this to pub/media. This helped my problem somewhat as I was filling 100GB of space in 30min and that changed to 60GB in 3-4 hours. Which was still crazy.

I will check in the morning (as its night my time) to see how the disk is filled. So far, it seemed to have slowed down, but I will report back in morning if disabling Magento_Csp improved it.

[mrtuvn]

As i remember this caused by high load plugin design load
Need some workarounds recheck disable this plugin and move to properly place with correct intercepts
https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/Theme/etc/di.xml#L109 <= default codebase
@hostep @amitvkhajanchi . This is confirmed issue but i don't remember exactly id

[amitvkhajanchi]

@hotstep @mrtuvn

Thanks for all the feed-back. I can confirm to you that disabling Magento_Csp module resolved the issue on Magento 2.4.1 version I have. My cache only grew to 600MB and is stable now compared to multiple GB previous days. This is inline with my experience on Magento 2.3.3. Thanks for the help and prompt feed-back. I really appreciate it!!!

[IbrahimS2]

Still, this is an issue on M2.4.2!

[DavorOptiweb]

We have same issue with 2 sites
One is 2.4.2
Second is 2.4.2-p1 (upgraded from 2.3.6)

It generated huge amount of redis cache keys and it all started with upgrade from 2.3.6 to 2.4.2

It goes even higher and server had to shut down due to out of memory (32G all together), so now I'm clearing redis every night to keep page alive

[mrtuvn]

#29964 (comment) @DavorOptiweb

[DavorOptiweb]

#29964 (comment) @DavorOptiweb

We have Magento_CSP disabled from start (never enabled)

[mrtuvn]

hmm! Interesting case
from above comment i see used csp module but not use redis but in your case that you claims enabled redis by default right ?
Maybe we need more investigate root cause on this why redis write so much data
Can you have monitor this case in latest code base (git 2.4-develop) ? I think this case require real data with big site for able reproduce

[IbrahimS2]

We have same issue with 2 sites
One is 2.4.2
Second is 2.4.2-p1 (upgraded from 2.3.6)

It generated huge amount of redis cache keys and it all started with upgrade from 2.3.6 to 2.4.2

It goes even higher and server had to shut down due to out of memory (32G all together), so now I'm clearing redis every night to keep page alive

@DavorOptiweb We have investigated this further, it appears what is consuming the memory in Redis is basically the 'page_cache', it stores the page source code which is consuming most of the memory, once we disabled/kept it off Redis it came down from 40GB to just up to 200-300MB memory usage.

The whole concept seems to be against the purpose.

[DavorOptiweb]

We are using Varnish for Page Cache.
Redis database for page cache is empty, only "default" cache is going to Redis.

If I have a look on Redis keys, this is what I see (most of "k" are block)

[IbrahimS2]

@DavorOptiweb Please share the cache declaration in your env.php file.