diff --git a/docs/source/index.md b/docs/source/index.md new file mode 100644 index 00000000000..feb2461ae2a --- /dev/null +++ b/docs/source/index.md @@ -0,0 +1,287 @@ +(home)= +# Welcome to the ExecuTorch Documentation + +**ExecuTorch** is PyTorch's solution to training and inference on the +Edge. + +## Key Value Propositions + +- **Portability:** Compatibility with a wide variety of computing + platforms, from high-end mobile phones to highly constrained + embedded systems and microcontrollers. +- **Productivity:** Enabling developers to use the same toolchains and + Developer Tools from PyTorch model authoring and conversion, to + debugging and deployment to a wide variety of platforms. +- **Performance:** Providing end users with a seamless and + high-performance experience due to a lightweight runtime and + utilizing full hardware capabilities such as CPUs, NPUs, and DSPs. + +ExecuTorch provides support for: + +* **Strong Model Support** LLMs (Large Language Models), + CV (Computer Vision), ASR (Automatic Speech Recognition), TTS (Text To Speech) +* **All Major Platforms** Android, Mac, Linux, Windows +* **Rich Acceleration Support** Apple, Arm, Cadence, MediaTek, + Qualcomm, Vulkan, XNNPACK + +### Documentation Navigation +#### Introduction +- [Overview](intro-overview) +- [How it Works](intro-how-it-works) +- [Getting Started with Architecture](getting-started-architecture) +- [Concepts](concepts) +#### Usage +- [Getting Started](getting-started) +- [Using Executorch Export](using-executorch-export) +- [Using Executorch on Android](using-executorch-android) +- [Using Executorch on iOS](using-executorch-ios) +- [Using Executorch with C++](using-executorch-cpp) +- [Runtime Integration](using-executorch-runtime-integration) +- [Troubleshooting](using-executorch-troubleshooting) +- [Building from Source](using-executorch-building-from-source) +- [FAQs](using-executorch-faqs) +#### Examples +- [Android Demo Apps](demo-apps-android.md) +- [iOS Demo Apps](demo-apps-ios.md) +#### Backends +- [Overview](backends-overview) +- [XNNPACK](backends-xnnpack) +- [Core ML](backends-coreml) +- [MPS](backends-mps) +- [Vulkan](backends-vulkan) +- [ARM Ethos-U](backends-arm-ethos-u) +- [Qualcomm](backends-qualcomm) +- [MediaTek](backends-mediatek) +- [Cadence](backends-cadence) +#### Developer Tools +- [Overview](devtools-overview) +- [Bundled IO](bundled-io) +- [ETRecord](etrecord) +- [ETDump](etdump) +- [Runtime Profiling](runtime-profiling) +- [Model Debugging](model-debugging) +- [Model Inspector](model-inspector) +- [Memory Planning Inspection](memory-planning-inspection) +- [Delegate Debugging](delegate-debugging) +- [Tutorial](devtools-tutorial) +#### Runtime +- [Overview](runtime-overview) +- [Extension Module](extension-module) +- [Extension Tensor](extension-tensor) +- [Running a Model (C++ Tutorial)](running-a-model-cpp-tutorial) +- [Backend Delegate Implementation and Linking](runtime-backend-delegate-implementation-and-linking) +- [Platform Abstraction Layer](runtime-platform-abstraction-layer) +#### Portable C++ Programming +- [PTE File Format](pte-file-format) +#### API Reference +- [Export to Executorch API Reference](export-to-executorch-api-reference) +- [Executorch Runtime API Reference](executorch-runtime-api-reference) +- [Runtime Python API Reference](runtime-python-api-reference) +- [API Life Cycle](api-life-cycle) +- [Javadoc](https://pytorch.org/executorch/main/javadoc/) +#### Quantization +- [Overview](quantization-overview) +#### Kernel Library +- [Overview](kernel-library-overview) +- [Custom ATen Kernel](kernel-library-custom-aten-kernel) +- [Selective Build](kernel-library-selective-build) +#### Working with LLMs +- [Llama](llm/llama) +- [Llama on Android](llm/llama-demo-android) +- [Llama on iOS](llm/llama-demo-ios) +- [Llama on Android via Qualcomm backend](llm/build-run-llama3-qualcomm-ai-engine-direct-backend) +- [Intro to LLMs in Executorch](llm/getting-started) +#### Backend Development +- [Delegates Integration](backend-delegates-integration) +- [XNNPACK Reference](backend-delegates-xnnpack-reference) +- [Dependencies](backend-delegates-dependencies) +- [Compiler Delegate and Partitioner](compiler-delegate-and-partitioner) +- [Debug Backend Delegate](debug-backend-delegate) +#### IR Specification +- [EXIR](ir-exir) +- [Ops Set Definition](ir-ops-set-definition) +#### Compiler Entry Points +- [Backend Dialect](compiler-backend-dialect) +- [Custom Compiler Passes](compiler-custom-compiler-passes) +- [Memory Planning](compiler-memory-planning) +#### Contributing +- [Contributing](contributing) + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Introduction +:hidden: + +intro-overview +intro-how-it-works +getting-started-architecture +concepts +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Usage +:hidden: + +getting-started +using-executorch-export +using-executorch-android +using-executorch-ios +using-executorch-cpp +using-executorch-runtime-integration +using-executorch-troubleshooting +using-executorch-building-from-source +using-executorch-faqs +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Examples +:hidden: + +demo-apps-android.md +demo-apps-ios.md +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Backends +:hidden: + +backends-overview +backends-xnnpack +backends-coreml +backends-mps +backends-vulkan +backends-arm-ethos-u +backends-qualcomm +backends-mediatek +backends-cadence +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Developer Tools +:hidden: + +devtools-overview +bundled-io +etrecord +etdump +runtime-profiling +model-debugging +model-inspector +memory-planning-inspection +delegate-debugging +devtools-tutorial +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Runtime +:hidden: + +runtime-overview +extension-module +extension-tensor +running-a-model-cpp-tutorial +runtime-backend-delegate-implementation-and-linking +runtime-platform-abstraction-layer +portable-cpp-programming +pte-file-format +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: API Reference +:hidden: + +export-to-executorch-api-reference +executorch-runtime-api-reference +runtime-python-api-reference +api-life-cycle +Javadoc <https://pytorch.org/executorch/main/javadoc/> +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Quantization +:hidden: + +quantization-overview +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Kernel Library +:hidden: + +kernel-library-overview +kernel-library-custom-aten-kernel +kernel-library-selective-build +``` + +```{toctree} +:glob: +:maxdepth: 2 +:caption: Working with LLMs +:hidden: + +Llama <llm/llama> +Llama on Android <llm/llama-demo-android> +Llama on iOS <llm/llama-demo-ios> +Llama on Android via Qualcomm backend <llm/build-run-llama3-qualcomm-ai-engine-direct-backend> +Intro to LLMs in Executorch <llm/getting-started> +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Backend Development +:hidden: + +backend-delegates-integration +backend-delegates-xnnpack-reference +backend-delegates-dependencies +compiler-delegate-and-partitioner +debug-backend-delegate +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: IR Specification +:hidden: + +ir-exir +ir-ops-set-definition +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Compiler Entry Points +:hidden: + +compiler-backend-dialect +compiler-custom-compiler-passes +compiler-memory-planning +``` + +```{toctree} +:glob: +:maxdepth: 1 +:caption: Contributing +:hidden: + +contributing +``` diff --git a/docs/source/index.rst b/docs/source/index.rst deleted file mode 100644 index 187a5300c58..00000000000 --- a/docs/source/index.rst +++ /dev/null @@ -1,378 +0,0 @@ -.. _home: - -Welcome to the ExecuTorch Documentation -======================================= - -.. important:: - v0.4.0 was the beta release of ExecuTorch. Starting from v0.4.0, the API - follows the `API Lifecycle and Deprecation Policy <api-life-cycle.html>`__, - and the ``.pte`` binary format complies with the `Runtime Compatibility - Policy - <https://github.com/pytorch/executorch/tree/main/runtime/COMPATIBILITY.md>`__. - This helps ensure that application developers can update to the latest - version of ExecuTorch without breaking existing integration code, in - accordance with these policies. If any issues arise or compatibility breaks - occur, please `report them in GitHub - <https://github.com/pytorch/executorch/issues/new/choose>`__. - - We welcome any feedback, suggestions, and bug reports from the community - to help us improve the technology. Please use the `PyTorch Forums - <https://discuss.pytorch.org/c/executorch>`__ for discussion and feedback - about ExecuTorch using the **ExecuTorch** category, and our `GitHub - repository <https://github.com/pytorch/executorch/issues>`__ for bug - reporting. - -.. raw:: html - - <div class="et-page-column-row"> - <div class="et-page-column1"><p><strong>ExecuTorch</strong> is a PyTorch platform that provides infrastructure to run PyTorch programs everywhere from AR/VR wearables to standard on-device iOS and Android mobile deployments. One of the main goals for ExecuTorch is to enable wider customization and deployment capabilities of the PyTorch programs.</p> - <p>ExecuTorch heavily relies on such PyTorch technologies as <a href="https://pytorch.org/docs/stable/torch.compiler.html">torch.compile</a> and <a href="https://pytorch.org/docs/main/export.html">torch.export</a>. If you are not familiar with these APIs, you might want to read about them in the PyTorch documentation before diving into the ExecuTorch documentation.</p></div> - <div class="et-page-column2"><img src="_static/img/ExecuTorch-Logo-cropped.svg" alt="ExecuTorch logo" title="ExecuTorch logo"></div> - </div> - -The ExecuTorch source is hosted on GitHub at -https://github.com/pytorch/executorch. - -Join us on `Discord <https://discord.com/invite/Dh43CKSAdc>`__ if you have questions -about ExecuTorch or would like to become a contributor! - -Getting Started -~~~~~~~~~~~~~~~ - -Topics in this section will help you get started with ExecuTorch. - -.. grid:: 3 - - .. grid-item-card:: :octicon:`file-code;1em` - What is ExecuTorch? - :img-top: _static/img/card-background.svg - :link: intro-overview.html - :link-type: url - - A gentle introduction to ExecuTorch. In this section, - you will learn about main features of ExecuTorch - and how you can use them in your projects. - - .. grid-item-card:: :octicon:`file-code;1em` - Getting started with ExecuTorch - :img-top: _static/img/card-background.svg - :link: getting-started.html - :link-type: url - - A step-by-step tutorial on how to get started with - ExecuTorch. - - .. grid-item-card:: :octicon:`file-code;1em` - ExecuTorch Llama - :img-top: _static/img/card-background.svg - :link: llm/llama.html - :link-type: url - - Learn about running Llama models via ExecuTorch - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Introduction - :hidden: - - intro-overview - intro-how-it-works - getting-started-architecture - concepts - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Usage - :hidden: - - getting-started - using-executorch-export - using-executorch-android - using-executorch-ios - using-executorch-cpp - using-executorch-runtime-integration - using-executorch-troubleshooting - using-executorch-building-from-source - using-executorch-faqs - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Examples - :hidden: - - Building an ExecuTorch Android Demo App <https://github.com/pytorch-labs/executorch-examples/tree/main/dl3/android/DeepLabV3Demo#executorch-android-demo-app> - demo-apps-ios.md - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Backends - :hidden: - - backends-overview - backends-xnnpack - backends-coreml - backends-mps - backends-vulkan - backends-arm-ethos-u - backends-qualcomm - backends-mediatek - backends-cadence - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Tutorials - :hidden: - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Developer Tools - :hidden: - - devtools-overview - bundled-io - etrecord - etdump - runtime-profiling - model-debugging - model-inspector - memory-planning-inspection - delegate-debugging - devtools-tutorial - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Runtime - :hidden: - - runtime-overview - extension-module - extension-tensor - running-a-model-cpp-tutorial - runtime-backend-delegate-implementation-and-linking - runtime-platform-abstraction-layer - portable-cpp-programming - pte-file-format - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: API Reference - :hidden: - - export-to-executorch-api-reference - executorch-runtime-api-reference - runtime-python-api-reference - api-life-cycle - Javadoc <https://pytorch.org/executorch/main/javadoc/> - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Quantization - :hidden: - - quantization-overview - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Kernel Library - :hidden: - - kernel-library-overview - kernel-library-custom-aten-kernel - kernel-library-selective-build - -.. toctree:: - :glob: - :maxdepth: 2 - :caption: Working with LLMs - :hidden: - - Llama <llm/llama> - Llama on Android <llm/llama-demo-android> - Llama on iOS <llm/llama-demo-ios> - Llama on Android via Qualcomm backend <llm/build-run-llama3-qualcomm-ai-engine-direct-backend> - Intro to LLMs in Executorch <llm/getting-started> - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Backend Development - :hidden: - - backend-delegates-integration - backend-delegates-xnnpack-reference - backend-delegates-dependencies - compiler-delegate-and-partitioner - debug-backend-delegate - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: IR Specification - :hidden: - - ir-exir - ir-ops-set-definition - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Compiler Entry Points - :hidden: - - compiler-backend-dialect - compiler-custom-compiler-passes - compiler-memory-planning - -.. toctree:: - :glob: - :maxdepth: 1 - :caption: Contributing - :hidden: - - contributing - -Tutorials and Examples -~~~~~~~~~~~~~~~~~~~~~~ - -Ready to experiment? Check out some of the -ExecuTorch tutorials. - -.. customcardstart:: - -.. customcarditem:: - :header: Exporting to ExecuTorch Tutorial - :card_description: A tutorial for exporting a model and lowering a it to be runnable on the ExecuTorch runtime. - :image: _static/img/generic-pytorch-logo.png - :link: tutorials/export-to-executorch-tutorial.html - :tags: Export,Delegation,Quantization - -.. customcarditem:: - :header: Running an ExecuTorch Model C++ Tutorial - :card_description: A tutorial for setting up memory pools, loading a model, setting inputs, executing the model, and retrieving outputs on device. - :image: _static/img/generic-pytorch-logo.png - :link: running-a-model-cpp-tutorial.html - :tags: - -.. customcarditem:: - :header: Simplified Runtime APIs Tutorial - :card_description: A simplified tutorial for executing the model on device. - :image: _static/img/generic-pytorch-logo.png - :link: extension-module.html - :tags: - -.. customcarditem:: - :header: Managing Tensor Memory in C++ Tutorial - :card_description: A tutorial for managing the dynamic memory when working with tensors. - :image: _static/img/generic-pytorch-logo.png - :link: extension-tensor.html - :tags: - -.. customcarditem:: - :header: Using the ExecuTorch Developer Tools to Profile a Model - :card_description: A tutorial for using the ExecuTorch Developer Tools to profile and analyze a model with linkage back to source code. - :image: _static/img/generic-pytorch-logo.png - :link: tutorials/devtools-integration-tutorial.html - :tags: devtools - -.. customcarditem:: - :header: Integrating and Running ExecuTorch on Apple Platforms - :card_description: A tutorial on integrating, using, and troubleshooting the ExecuTorch runtime on iOS. - :image: _static/img/generic-pytorch-logo.png - :link: apple-runtime.html - :tags: iOS, macOS - -.. customcarditem:: - :header: Building an ExecuTorch iOS Demo App - :card_description: A demo tutorial that explains how to build ExecuTorch into iOS frameworks and run an iOS app. - :image: _static/img/demo_ios_app.jpg - :link: demo-apps-ios.html - :tags: Delegation,iOS - -.. customcarditem:: - :header: Building an ExecuTorch Android Demo App - :card_description: A demo tutorial that explains how to build ExecuTorch into a JNI library and build an Android app. - :image: _static/img/android_app.png - :link: demo-apps-android.html - :tags: Delegation,Android - -.. customcarditem:: - :header: Lowering a Model as a Delegate - :card_description: An end-to-end example showing how to lower a model as a delegate - :image: _static/img/generic-pytorch-logo.png - :link: examples-end-to-end-to-lower-model-to-delegate.html - :tags: Export,Delegation - -.. - First-party backends that are good intros for readers. - -.. customcarditem:: - :header: Building and Running ExecuTorch with XNNPACK Backend - :card_description: A demo tutorial for lowering and exporting models with the XNNPACK Backend - :image: _static/img/generic-pytorch-logo.png - :link: tutorial-xnnpack-delegate-lowering.html - :tags: Export,Backend,Delegation,Quantization,XNNPACK - -.. customcarditem:: - :header: Building and Running ExecuTorch with Vulkan Backend - :card_description: A tutorial that walks you through the process of building ExecuTorch with Vulkan Backend - :image: _static/img/generic-pytorch-logo.png - :link: backends-vulkan.html - :tags: Export,Backend,Delegation,Vulkan - -.. - Alphabetical by backend name. Be sure to keep the same order in the Tutorials - toctree entry above. - -.. customcarditem:: - :header: Building and Running ExecuTorch with ARM Ethos-U Backend - :card_description: A tutorial that walks you through the process of building ExecuTorch with ARM Ethos-U Backend - :image: _static/img/generic-pytorch-logo.png - :link: executorch-arm-delegate-tutorial.html - :tags: Export,Backend,Delegation,ARM,Ethos-U - -.. customcarditem:: - :header: Building and Running ExecuTorch with CoreML Backend - :card_description: A tutorial that walks you through the process of building ExecuTorch with CoreML Backend - :image: _static/img/generic-pytorch-logo.png - :link: backends-coreml.html - :tags: Export,Backend,Delegation,CoreML - -.. customcarditem:: - :header: Building and Running ExecuTorch with MediaTek Backend - :card_description: A tutorial that walks you through the process of building ExecuTorch with MediaTek Backend - :image: _static/img/generic-pytorch-logo.png - :link: backends-mediatek-backend.html - :tags: Export,Backend,Delegation,MediaTek - -.. customcarditem:: - :header: Building and Running ExecuTorch with MPS Backend - :card_description: A tutorial that walks you through the process of building ExecuTorch with MPSGraph Backend - :image: _static/img/generic-pytorch-logo.png - :link: backends-mps.html - :tags: Export,Backend,Delegation,MPS,MPSGraph - -.. customcarditem:: - :header: Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend - :card_description: A tutorial that walks you through the process of building ExecuTorch with Qualcomm AI Engine Direct Backend - :image: _static/img/generic-pytorch-logo.png - :link: backends-qualcomm.html - :tags: Export,Backend,Delegation,QNN - -.. customcarditem:: - :header: Building and Running ExecuTorch on Xtensa HiFi4 DSP - :card_description: A tutorial that walks you through the process of building ExecuTorch for an Xtensa Hifi4 DSP using custom operators - :image: _static/img/generic-pytorch-logo.png - :link: backends-cadence.html - :tags: Export,Custom-Operators,DSP,Xtensa - -.. customcardend::