You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update on "[ET-VK][AOT] Serialize constant tensors via NamedDataMap"
Summary:
When exporting models to Vulkan backend, save constant tensors in the NamedDataMap instead of the constant data section of the delegate header.
## Motivation
Prevent screen blackout (Llama 3.2 1B) / device crash (Llama 3.2 3B) when running Llama 3.2 models on Samsung Galaxy S24. This behaviour is related to high peak memory usage when loading the model. For more information, see the top diff/PR in the stack.
## Context
This change is based on the equivalent change D70315207/#9153 in XNNPACK.
Test Plan:
## Memory Comparison with/without NamedDataMap
Measured VmRss using
```
uint64_t getVmRssInKB() {
std::ifstream statusFile("/proc/self/status");
std::string l, num;
while (std::getline(statusFile, l)) {
if (l.substr(0, 5) == "VmRSS") {
size_t pos = l.find_first_of("0123456789");
num = l.substr(pos);
break;
}
}
uint64_t vmRssInKB = std::stoi(num);
return vmRssInKB;
}
```
P1908019767 (Meta only)
Excerpt:
```
Log 1 | Log 2
--------------------------------------------------|--------------------------------------------------
Memory usage before model compilation: 1115416 KB | Memory usage before model compilation: 1919228 KB
Memory usage after graph building: 1924340 KB | Memory usage after graph building: 1924256 KB
Memory usage after graph preparation: 1798968 KB | Memory usage after graph preparation: 1782464 KB
Memory usage prepack start: 1798968 KB | Memory usage prepack start: 1781968 KB
Memory usage after prepack operations: 1271924 KB | Memory usage after prepack operations: 1653496 KB
```
Differential Revision: [D80460034](https://our.internmc.facebook.com/intern/diff/D80460034)
[ghstack-poisoned]
0 commit comments