Skip to content

Document common CV patterns #8831

Open
Open
@GregoryComer

Description

@GregoryComer

📚 The doc issue

There are a number of CV-specific tips, tricks, and techniques that are helpful for deploying CV models on ExecuTorch. From internal adoption, we've seen that CV models make up a sizable portion of on-device use cases, so I think it's worth having some docs to cover a few things. It will also allow me to link to it in the getting started doc (which uses torchvision v2) to avoid needing to explain input transformations and such there.

I'm not entirely sure where to put this yet. I don't necessarily want to stick it under Usage/, since I'm trying to keep that higher-level. Maybe we need a new section for moderate to advanced usage docs for this type of thing. We can also maybe stick docs on export compliance tricks (#8731) and model optimization techniques there, as well.

Things to cover:

  • Wrapping a model with common input transformations:
    • Image resize
    • Normalization
    • Dtype conversion for u8 input
  • Input and output format for common classes of models
    • Channels first vs channels last format
    • Examples of loading images and converting to tensor format on Android + iOS
    • Briefly cover interpreting output class scores, masks, instance segmentation outputs, etc. Don't go into too much detail, but give a quick overview and maybe some Pytorch or torchvision links.

Suggest a potential alternative/fix

No response

cc @mergennachin @byjlw

Metadata

Metadata

Assignees

Labels

module: docIssues related to documentation, both in docs/ and inlined in codemodule: user experienceIssues related to reducing friction for userstriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions