Skip to content

[RFC] Canonicalize python file structure #8699

Open
@larryliu0820

Description

@larryliu0820

🚀 The feature, motivation and pitch

Blocking #2871

Context

Currently the file structure looks like this:

executorch
├── backends
...
├── exir
├── extension
...
├── runtime
├── setup.py
...
└── version.txt

Problem

  • Since we don't have an executorch/ directory, we can't add code into executorch root module, we will have to do things like import executorch.exir etc. In the future we want to do from executorch import <top level API>
  • We want to have a clean separation between python code and C++ code, for easier build system management.
    • E.g., we don't want to build python code in GH, so it will be easier to configure the internal repo to not map TARGETS files to GH.
  • Editable mode install couldn't find the top level module correctly and thus failed to install.
    • This is because when setuptools tries to map the module namespace to actual files, it will do the following mapping: 'executorch/backens.apple': ['<path to local executorch repo>/executorch/backends/apple']. setuptools couldn't define a root level executorch module as well because it only lookup folders under executorch/. See NAMESPACE section in this generated code, instead we want this code.

Proposal

Move all python code into executorch/ and use that as our root level module. Keep all native source files intact.

executorch
├── backends
│   ├── apple // no python code
│   ├── arm
├── executorch // only python code
│   ├── backends
│   ├── exir
│   └── extension
...
└── version.txt

This way we can build a self-contained executorch module which works for editable mode.

Migrations

If we go down this route, we will have to migrate python targets to executorch/ and change internal usage. I don't expect this to affect pip users since they will just do the same from executorch.exir import X.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

Metadata

Metadata

Assignees

Labels

enhancementNot as big of a feature, but technically not a bug. Should be easy to fixmodule: armIssues related to arm backendtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

Status

Backlog

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions