-
Notifications
You must be signed in to change notification settings - Fork 371
Closed
Labels
Story: Dynamo Compile ImprovementsIssues relating to improvement of the Dynamo compile pathIssues relating to improvement of the Dynamo compile pathfeature requestNew feature or requestNew feature or request
Description
Context
Currently, the torch.compile
backend for Torch-TensorRT requires @fake_tensor_unsupported
due to a few functions which cannot handle fake tensors. This support needs to be updated in order to support fake tensors on the path to full-scale dynamic shape support.
Key Consideration: Is it possible to support fake tensors while preserving the notion of "constant" tensors in TRT?
Proposed Changes
The to_numpy
function will need to be updated to take a scalar:
def to_numpy(tensor: Optional[torch.Tensor]) -> Optional[np.ndarray]: |
The create_constant
function will need to be updated to not transform scalars into tensors:
def create_constant( |
Special considerations must be made for constant tensors with more than one element.
Metadata
Metadata
Assignees
Labels
Story: Dynamo Compile ImprovementsIssues relating to improvement of the Dynamo compile pathIssues relating to improvement of the Dynamo compile pathfeature requestNew feature or requestNew feature or request