Qualcomm AI Engine Direct - multi-method support

[haowhsu-quic]

Summary

• refactor to adopt multi-method change
• framework patch to meet multi-method use case

Test plan

python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedUtils.test_qnn_backend_multi_graphs -s $device_sn -b build-android -m SM8750

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10584

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 New Failures

As of commit 30883fd with merge base 48ad9f6 ():

NEW FAILURES - The following jobs have failed:

• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 8b9261964e6f35861f8c195a8c3f83b070b36637827e0e5d9292fb3031573168 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command docker exec -t 4cb70f16f1908e5e136c052d42160f55be38bf1790a0544bf215219916563224 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t 6073b9ac9c749fbe46126e13e495ceb08897f784ce7f10676493c20b2baf4aa2 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t 336f7573fc91981785c1149fe8ef7b2898a7615733cdcc338ba5fc3369d3698f /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 623a35e643b7a185768703fccc1085317bad9a269a3d10077d9744fb25123de4 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  RuntimeError: Command docker exec -t 894bb7059a5348f1ba533ce3ef289f91cff177a59206d918ebfa050403201797 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  test_llava_export
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5d7092fc8eefc76c292b0ee466e9399b18384c803d9b611ded4ec875b7e7ec4f /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  exir/backend/test/test_to_backend_multi_method.py::TestToBackendMultiMethod::test_multi_method_to_backend_two_methods_different_backends
• pull / unittest / macos / macos-job (gh)
  exir/backend/test/test_to_backend_multi_method.py::TestToBackendMultiMethod::test_multi_method_to_backend_two_methods_different_backends
• pull / unittest-editable / linux / linux-job (gh)
  exir/backend/test/test_to_backend_multi_method.py::TestToBackendMultiMethod::test_multi_method_to_backend_two_methods_different_backends
• pull / unittest-editable / macos / macos-job (gh)
  exir/backend/test/test_to_backend_multi_method.py::TestToBackendMultiMethod::test_multi_method_to_backend_two_methods_different_backends

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[haowhsu-quic]

@pytorchbot label "release notes: qualcomm"

[cccclai]

Can you share what issues you run into?

[haowhsu-quic]

The scenario happens when there are common nodes shared by multiple delegated subgraphs (e.g. frequency sin/cos in sharded llama):
The example graph looks like below:

When replacing submodule fused_qnn_1, the original name finding mechanism are trying to match between:

• submodule_program.graph_signature.user_inputs: ('aten_mean_dim', 'aten_select_int', 'aten_select_int_1')
• call_submodule_node.all_input_nodes: [aten_mean_dim, getitem_1, getitem_2]

Which makes getitem node dangling as following, since the names could not be correctly mapped:

The patch here is trying to find the original graph with real output names and use index of getitem to have them in correct order.

And I think another issue is about validation in _unsafe_adjust_original_program. Since the partitioned sub graphs in multi-method scenario have already been turned into submodules (like first diagram). That behavior will make original_program._validate() fail in _unsafe_adjust_original_program.
This does not happen in single method lowering because sub graphs are turned into submodules one by one and replaced into executorch_call_delegate.
But I cannot find an appropriate way to pass official CI, could you give me some hint? thank you!