Skip to content

Commit 1fc589b

Browse files
authored
fix user_guide and tutorial docs (#2854)
1 parent 431e9a9 commit 1fc589b

File tree

6 files changed

+25
-25
lines changed

6 files changed

+25
-25
lines changed

docsrc/tutorials/notebooks.rst

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,13 @@ and running it to test the speedup obtained.
2323
* `Torch-TensorRT Getting Started - CitriNet <https://github.com/pytorch/TensorRT/blob/master/notebooks/CitriNet-example.ipynb>`_
2424

2525

26-
Compiling EfficentNet with Torch-TensorRT
26+
Compiling EfficientNet with Torch-TensorRT
2727
********************************************
2828

29-
EfficentNet is a feedforward CNN designed to achieve better performance and accuracy than alternative architectures
29+
EfficientNet is a feedforward CNN designed to achieve better performance and accuracy than alternative architectures
3030
by using a "scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient".
3131

32-
This notebook demonstrates the steps for optimizing a pretrained EfficentNet model with Torch-TensorRT,
32+
This notebook demonstrates the steps for optimizing a pretrained EfficientNet model with Torch-TensorRT,
3333
and running it to test the speedup obtained.
3434

3535
* `Torch-TensorRT Getting Started - EfficientNet-B0 <https://github.com/pytorch/TensorRT/blob/master/notebooks/EfficientNet-example.ipynb>`_
@@ -43,7 +43,7 @@ This way, the model learns an inner representation of the English language that
4343
features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train
4444
a standard classifier using the features produced by the BERT model as inputs." (https://huggingface.co/bert-base-uncased)
4545

46-
This notebook demonstrates the steps for optimizing a pretrained EfficentNet model with Torch-TensorRT,
46+
This notebook demonstrates the steps for optimizing a pretrained EfficientNet model with Torch-TensorRT,
4747
and running it to test the speedup obtained.
4848

4949
* `Masked Language Modeling (MLM) with Hugging Face BERT Transformer <https://github.com/pytorch/TensorRT/blob/master/notebooks/Hugging-Face-BERT.ipynb>`_
@@ -73,7 +73,7 @@ Using Dynamic Shapes with Torch-TensorRT
7373

7474
Making use of Dynamic Shaped Tensors in Torch TensorRT is quite simple. Let's say you are
7575
using the ``torch_tensorrt.compile(...)`` function to compile a torchscript module. One
76-
of the args in this function in this function is ``input``: which defines an input to a
76+
of the args in this function is ``input``: which defines an input to a
7777
module in terms of expected shape, data type and tensor format: ``torch_tensorrt.Input.``
7878

7979
For the purposes of this walkthrough we just need three kwargs: `min_shape`, `opt_shape`` and `max_shape`.
@@ -96,8 +96,8 @@ In this example, we are going to use a simple ResNet model to demonstrate the us
9696
Using the FX Frontend with Torch-TensorRT
9797
********************************************
9898

99-
The purpose of this example is to demostrate the overall flow of lowering a PyTorch model to TensorRT
100-
conveniently with using FX.
99+
The purpose of this example is to demonstrate the overall flow of lowering a PyTorch model to TensorRT
100+
conveniently using FX.
101101

102102
* `Using the FX Frontend with Torch-TensorRT <https://github.com/pytorch/TensorRT/blob/master/notebooks/getting_started_with_fx_path_lower_to_trt.ipynb>`_
103103

docsrc/tutorials/serving_torch_tensorrt_with_triton.rst

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@ Serving a Torch-TensorRT model with Triton
44
==========================================
55

66
Optimization and deployment go hand in hand in a discussion about Machine
7-
Learning infrastructure. Once network level optimzation are done
7+
Learning infrastructure. Once network level optimization are done
88
to get the maximum performance, the next step would be to deploy it.
99

10-
However, serving this optimized model comes with it's own set of considerations
11-
and challenges like: building an infrastructure to support concorrent model
10+
However, serving this optimized model comes with its own set of considerations
11+
and challenges like: building an infrastructure to support concurrent model
1212
executions, supporting clients over HTTP or gRPC and more.
1313

1414
The `Triton Inference Server <https://github.com/triton-inference-server/server>`__
@@ -67,7 +67,7 @@ highly recommend to checking our `Github
6767
Repository <https://github.com/triton-inference-server>`__.
6868

6969
To use Triton, we need to make a model repository. A model repository, as the
70-
name suggested, is a repository of the models the Inference server hosts. While
70+
name suggests, is a repository of the models the Inference server hosts. While
7171
Triton can serve models from multiple repositories, in this example, we will
7272
discuss the simplest possible form of the model repository.
7373

@@ -204,7 +204,7 @@ Lastly, we send an inference request to the Triton Inference Server.
204204
inference_output = results.as_numpy('output__0')
205205
print(inference_output[:5])
206206

207-
The output of the same should look like below:
207+
The output should look like below:
208208

209209
::
210210

docsrc/user_guide/dynamic_shapes.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ Dynamic shapes with Torch-TensorRT
66
By default, you can run a pytorch model with varied input shapes and the output shapes are determined eagerly.
77
However, Torch-TensorRT is an AOT compiler which requires some prior information about the input shapes to compile and optimize the model.
88
In the case of dynamic input shapes, we must provide the (min_shape, opt_shape, max_shape) arguments so that the model can be optimized for
9-
these range of input shapes. An example usage of static and dynamic shapes is as follows.
9+
this range of input shapes. An example usage of static and dynamic shapes is as follows.
1010

11-
NOTE: The following code uses Dynamo Frontend. Incase of Torchscript Frontend, please swap out ``ir=dynamo`` with ``ir=ts`` and the behavior is exactly the same.
11+
NOTE: The following code uses Dynamo Frontend. In case of Torchscript Frontend, please swap out ``ir=dynamo`` with ``ir=ts`` and the behavior is exactly the same.
1212

1313
.. code-block:: python
1414

docsrc/user_guide/ptq.rst

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@ Users writing TensorRT applications are required to setup a calibrator class whi
1313
the TensorRT calibrator. With Torch-TensorRT we look to leverage existing infrastructure in PyTorch to make implementing
1414
calibrators easier.
1515

16-
LibTorch provides a ``DataLoader`` and ``Dataset`` API which steamlines preprocessing and batching input data.
16+
LibTorch provides a ``DataLoader`` and ``Dataset`` API which streamlines preprocessing and batching input data.
1717
These APIs are exposed via both C++ and Python interface which makes it easier for the end user.
1818
For C++ interface, we use ``torch::Dataset`` and ``torch::data::make_data_loader`` objects to construct and perform pre-processing on datasets.
1919
The equivalent functionality in python interface uses ``torch.utils.data.Dataset`` and ``torch.utils.data.DataLoader``.
2020
This section of the PyTorch documentation has more information https://pytorch.org/tutorials/advanced/cpp_frontend.html#loading-data and https://pytorch.org/tutorials/recipes/recipes/loading_data_recipe.html.
2121
Torch-TensorRT uses Dataloaders as the base of a generic calibrator implementation. So you will be able to reuse or quickly
22-
implement a ``torch::Dataset`` for your target domain, place it in a DataLoader and create a INT8 Calibrator
23-
which you can provide to Torch-TensorRT to run INT8 Calibration during compliation of your module.
22+
implement a ``torch::Dataset`` for your target domain, place it in a DataLoader and create an INT8 Calibrator
23+
which you can provide to Torch-TensorRT to run INT8 Calibration during compilation of your module.
2424

2525
.. _writing_ptq_cpp:
2626

@@ -108,7 +108,7 @@ Next we create a calibrator from the ``calibration_dataloader`` using the calibr
108108

109109
Here we also define a location to write a calibration cache file to which we can use to reuse the calibration data without needing the dataset and whether or not
110110
we should use the cache file if it exists. There also exists a ``torch_tensorrt::ptq::make_int8_cache_calibrator`` factory which creates a calibrator that uses the cache
111-
only for cases where you may do engine building on a machine that has limited storage (i.e. no space for a full dataset) or to have a simpiler deployment application.
111+
only for cases where you may do engine building on a machine that has limited storage (i.e. no space for a full dataset) or to have a simpler deployment application.
112112

113113
The calibrator factories create a calibrator that inherits from a ``nvinfer1::IInt8Calibrator`` virtual class (``nvinfer1::IInt8EntropyCalibrator2`` by default) which
114114
defines the calibration algorithm used when calibrating. You can explicitly make the selection of calibration algorithm like this:
@@ -118,7 +118,7 @@ defines the calibration algorithm used when calibrating. You can explicitly make
118118
// MinMax Calibrator is geared more towards NLP tasks
119119
auto calibrator = torch_tensorrt::ptq::make_int8_calibrator<nvinfer1::IInt8MinMaxCalibrator>(std::move(calibration_dataloader), calibration_cache_file, true);
120120

121-
Then all thats required to setup the module for INT8 calibration is to set the following compile settings in the `torch_tensorrt::CompileSpec` struct and compiling the module:
121+
Then all that's required to setup the module for INT8 calibration is to set the following compile settings in the `torch_tensorrt::CompileSpec` struct and compiling the module:
122122

123123
.. code-block:: c++
124124

docsrc/user_guide/runtime.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Deploying Torch-TensorRT Programs
44
====================================
55

66
After compiling and saving Torch-TensorRT programs there is no longer a strict dependency on the full
7-
Torch-TensorRT library. All that is required to run a compiled program is the runtime. There are therfore a couple
7+
Torch-TensorRT library. All that is required to run a compiled program is the runtime. There are therefore a couple
88
options to deploy your programs other than shipping the full Torch-TensorRT compiler with your applications.
99

1010
Torch-TensorRT package / libtorchtrt.so
@@ -24,7 +24,7 @@ programs just as you would otherwise via PyTorch API.
2424

2525
.. note:: If you are using the standard distribution of PyTorch in Python on x86, likely you will need the pre-cxx11-abi variant of ``libtorchtrt_runtime.so``, check :ref:`Installation` documentation for more details.
2626

27-
.. note:: If you are linking ``libtorchtrt_runtime.so``, likely using the following flags will help ``-Wl,--no-as-needed -ltorchtrt -Wl,--as-needed`` as theres no direct symbol dependency to anything in the Torch-TensorRT runtime for most Torch-TensorRT runtime applications
27+
.. note:: If you are linking ``libtorchtrt_runtime.so``, likely using the following flags will help ``-Wl,--no-as-needed -ltorchtrt -Wl,--as-needed`` as there's no direct symbol dependency to anything in the Torch-TensorRT runtime for most Torch-TensorRT runtime applications
2828

2929
An example of how to use ``libtorchtrt_runtime.so`` can be found here: https://github.com/pytorch/TensorRT/tree/master/examples/torchtrt_runtime_example
3030

@@ -33,7 +33,7 @@ Plugin Library
3333

3434
In the case you use Torch-TensorRT as a converter to a TensorRT engine and your engine uses plugins provided by Torch-TensorRT, Torch-TensorRT
3535
ships the library ``libtorchtrt_plugins.so`` which contains the implementation of the TensorRT plugins used by Torch-TensorRT during
36-
compilation. This library can be ``DL_OPEN`` or ``LD_PRELOAD`` similar to other TensorRT plugin libraries.
36+
compilation. This library can be ``DL_OPEN`` or ``LD_PRELOAD`` similarly to other TensorRT plugin libraries.
3737

3838
Multi Device Safe Mode
3939
---------------
@@ -60,7 +60,7 @@ doubles as a context manager.
6060
TensorRT requires that each engine be associated with the CUDA context in the active thread from which it is invoked.
6161
Therefore, if the device were to change in the active thread, which may be the case when invoking
6262
engines on multiple GPUs from the same Python process, safe mode will cause Torch-TensorRT to display
63-
an alert and switch GPUs accordingly. If safe mode were not enabled, there could be a mismatch in the engine
63+
an alert and switch GPUs accordingly. If safe mode is not enabled, there could be a mismatch in the engine
6464
device and CUDA context device, which could lead the program to crash.
6565

6666
One technique for managing multiple TRT engines on different GPUs while not sacrificing performance for

docsrc/user_guide/using_dla.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ DLA
77

88
NOTE: DLA supports fp16 and int8 precision only.
99

10-
Using DLA with torchtrtc
10+
Using DLA with `torchtrtc`
1111

1212
.. code-block:: shell
1313
@@ -41,7 +41,7 @@ Using DLA in a python application
4141
compile_spec = {
4242
"inputs": [torch_tensorrt.Input(self.input.shape)],
4343
"device": torch_tensorrt.Device("dla:0", allow_gpu_fallback=True),
44-
"enalbed_precisions": {torch.half},
44+
"enabled_precisions": {torch.half},
4545
}
4646
4747
trt_mod = torch_tensorrt.compile(self.scripted_model, compile_spec)

0 commit comments

Comments
 (0)