-
Notifications
You must be signed in to change notification settings - Fork 619
Run iPhone benchmark on private devices #10380
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10380
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New FailuresAs of commit ea1cab8 with merge base 59870c5 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
iOS 15. Is it a typo, you mean iOS 18? |
Yeah, we have iPhone 15 iOS 18 and iPhone 15 iOS 17.2.1 |
After looking at the test run on the dashboard. The devices shows up as |
Same as #10192, but this is for the private iPhone 15 devices. I will need to follow on this with another PR to get the device pool name / id and maybe the device id
Testing
https://hud.pytorch.org/benchmark/llms?startTime=Wed%2C%2016%20Apr%202025%2004%3A22%3A32%20GMT&stopTime=Wed%2C%2023%20Apr%202025%2004%3A22%3A32%20GMT&granularity=day&lBranch=apple-private-devices&lCommit=be72291c9b643581fedafaf8711d07fd5542b62d&rBranch=apple-private-devices&rCommit=be72291c9b643581fedafaf8711d07fd5542b62d&repoName=pytorch%2Fexecutorch&benchmarkName=&modelName=All%20Models&backendName=All%20Backends&modeName=All%20Modes&dtypeName=All%20DType&deviceName=All%20Devices&archName=All%20Platforms