microsoft_defender_endpoint.vulnerability and m365_defender.vulnerability data streams consume huge memory while retrieving larger workloads.

The current approach retrieves all software vulnerabilities in user's environment by pulling data from 3 APIs before combining them, which isn't scalable.
APIs endpoints used inside the CEL program are:

• /api/vulnerabilities/machinesVulnerabilities: Data retrieved is proportional to number of machines and softwares installed on them.
• /api/machines: Data retrieved is proportional to number of machines.
• /api/vulnerabilities: Data retrieved contains all known vulnerabilities.

Even for medium workloads (few thousand machines and/or softwares installed), the CEL input can hang without indexing the data.
Instead of pulling data from 3 APIs and combining them inside CEL program, a single API: Export software vulnerabilities assessment (via files) can fetch all software vulnerabilities from user's machines, and is the preferred approach:

This API solution enables pulling larger amounts of data faster and more reliably. Via-files is recommended for large organizations, with more than 100-K devices.

Moving to this new API there will be some fields missed from previous approach (event combining 3 APIs). But these missing fields wouldn't break the Security Vulnerabilities workflow. See #15521 (comment) for details on missing fields.