🐛 [Bug] Half precision index tensor detected when converting mistralai/Mistral-7B-v0.3 #2958
Description
Bug Description
mistralai/Mistral-7B-v0.3 is not fully supported because the index inputs to torch.ops.aten.index.Tensor are half-precision according to the fake tensor metadata.
To Reproduce
torch.compile and run inference on the mistralai/Mistral-7B-v0.3 model
Expected behavior
The graph consists of 2895 Total Operators, of which 2831 operators are supported, 97.79% coverage
The following ops are currently unsupported or excluded from conversion, and are listed with their op-count in the graph:
torch.ops.aten.index.Tensor: 64
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0):
2.3.0 - PyTorch Version (e.g. 1.0):
2.3.1+cu121 - Transformers Version:
4.40.2