❓ [Question] Building torch_tensorrt.lib on Windows

[jonahclarsen]

❓ Question

I am wondering how to build the torch_tensorrt.lib on Windows.

What you have already tried

I have followed #960 and #856 (with the same WORKSPACE as the latter) and managed to successfully build torch_tensorrt.dll. However, I need the .lib file in order to compile my Libtorch program. I tried linking to some of the .lib files that were created already (like bazel-out\x64_windows-opt\bin\cpp\torch_tensorrt.lo.lib), but that didn't work. I expect it's a fairly simple bazel command, but I have no idea where to put it.

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• PyTorch Version (e.g., 1.0): 1.10.0 (release)
• CPU Architecture: x86-64
• OS (e.g., Linux): Windows 10
• How you installed PyTorch (conda, pip, libtorch, source): libtorch from pytorch.org
• Build command you used (if compiling from source): bazel build //:libtorchtrt --compilation_mode opt
• CUDA version: 11.3
• Any other relevant information: Using VS2019

Additional context

My libtorch program runs fine even if I include the torch-tensorrt headers, but throws the following errors as soon as I try to use torch_tensorrt::torchscript::CompileSpec and call torch_tensorrt::torchscript::compile:
Error LNK1120 2 unresolved externals Omkar 1.10.0+cu113 B:\Programming_Current Projects\HelloLibTorch\x64\Release\HelloTorch.exe 1

Error LNK2019 unresolved external symbol "public: __cdecl torch_tensorrt::torchscript::CompileSpec::CompileSpec(class std::vector<class std::vector<__int64,class std::allocator<__int64> >,class std::allocator<class std::vector<__int64,class std::allocator<__int64> > > >)" (??0CompileSpec@torchscript@torch_tensorrt@@qeaa@V?$vector@V?$vector@_JV?$allocator@_J@std@@@std@@v?$allocator@V?$vector@_JV?$allocator@_J@std@@@std@@@2@@std@@@z) referenced in function main Omkar 1.10.0+cu113 B:\Programming_Current Projects\HelloLibTorch\main.obj 1

Error LNK2019 unresolved external symbol "struct torch::jit::Module __cdecl torch_tensorrt::torchscript::compile(struct torch::jit::Module const &,struct torch_tensorrt::torchscript::CompileSpec)" (?compile@torchscript@torch_tensorrt@@ya?AUModule@jit@torch@@AEBU345@UCompileSpec@12@@z) referenced in function main Omkar 1.10.0+cu113 B:\Programming_Current Projects\HelloLibTorch\main.obj 1

[jonahclarsen]

I managed to create the .lib file from the .dll following these instructions: https://web.archive.org/web/20140219172454/https://adrianhenke.wordpress.com/2008/12/05/create-lib-file-from-dll/ (adding /MACHINE:X64 to the lib command), but the .dll didn't contain the torchscript namespace and its functions/classes.

I found that the torchscript namespace was in cpp_objs\torch_tensorrt\compile_spec.obj, but when I tried turning the .objs in that folder into a .lib (with VS tools' "lib": https://stackoverflow.com/questions/31763558/how-to-build-static-and-dynamic-libraries-from-obj-files-for-visual-c) and linking it in my probject, it just created a bunch of new 'unresolved external symbol' linker errors.

Any suggestions on how to build .lib and .dll files for all of torch_tensorrt, including compile_spec, ptq, etc.?

@narendasan perhaps you would have some insight into this? Thanks!

[narendasan]

Are the symbols from the top level namespace available? Like torch_tensorrt::Input? Unfortunately I don't have a ton of experience with windows development.

Perhaps bazel can directly generate the .lib for you. The final library targets in the repo are located at //cpp/lib/BUILD. There is already a dll target, perhaps you could also make a .lib target as well. I think all you would need to do is copy the dll target but change the file type to .lib. Also I was looking at the cc_binary documentation and it seems there is a field called win_def_file

The Windows DEF file to be passed to linker.

This attribute should only be used when Windows is the target platform. It can be used to export symbols during linking a shared library.

Not sure if this would help here.

[jonahclarsen]

@narendasan Even torch_tensorrt::Input throws an 'unresolved external symbol' linker error.

I tried adding a .lib target to cpp/lib/BUILD just under the .dll target, but it doesn't do anything.

There is a 4KB torch_tensorrt.def file in cpp/lib, but when I added it as shown below (I tried with both the .dll and .lib targets):

cc_binary(
name = "torch_tensorrt.dll",
srcs = [],
linkshared = True,
linkstatic = True,
deps = [
"//cpp:torch_tensorrt",
],
win_def_file="torch_tensorrt.def",
)

cc_binary(
name = "torch_tensorrt.lib",
srcs = [],
linkshared = True,
linkstatic = True,
deps = [
"//cpp:torch_tensorrt",
],
win_def_file="torch_tensorrt.def",
)

I got the following error:
INFO: Analyzed target //:libtorchtrt (1 packages loaded, 4 targets configured). INFO: Found 1 target... ERROR: C:/users/jonah/downloads/torch-tensorrt-1.0.0/cpp/lib/BUILD:34:10: Linking cpp/lib/torch_tensorrt.dll failed: missing input file '//cpp/lib:torch_tensorrt.def' Target //:libtorchtrt failed to build Use --verbose_failures to see the command lines of failed build steps. ERROR: C:/users/jonah/downloads/torch-tensorrt-1.0.0/cpp/lib/BUILD:34:10 Linking cpp/lib/torch_tensorrt.dll failed: 1 input file(s) do not exist INFO: Elapsed time: 0.294s, Critical Path: 0.01s INFO: 1 process: 1 internal. FAILED: Build did NOT complete successfully

And it wouldn't let me put the full directory as it threw an error if the filename included a ':', '/', or ''.

Thanks for your help so far.

[narendasan]

So the lib target will not get included in the package builder target //:libtorchtrt (defined in //BUILD). Probably its best to start by trying to build //cpp/lib:torch_tensorrt.lib before dealing with packaging.

I was reading a bit more on def files and it seems like it's not really necessary, but we might need to change the TORCHTRT_API macro (https://github.com/NVIDIA/Torch-TensorRT/blob/master/cpp/include/torch_tensorrt/macros.h) to add __declspec(dllexport) to the source: https://docs.microsoft.com/en-us/cpp/build/exporting-from-a-dll-using-declspec-dllexport?view=msvc-170

[narendasan]

Re the missing file error, that is because bazel expects a def file in //cpp/lib

[jonahclarsen]

@narendasan there is a def file in //cpp/lib

[narendasan]

Hmm and this is the source //cpp/lib and not like //bazel-torch-tensorrt/cpp/lib? Maybe try clearing the cache? bazel clean --expunge

…

________________________________ From: Jonah ***@***.***> Sent: Sunday, May 1, 2022 3:30:48 PM To: NVIDIA/Torch-TensorRT ***@***.***> Cc: Naren Dasan ***@***.***>; Mention ***@***.***> Subject: Re: [NVIDIA/Torch-TensorRT] ❓ [Question] Building torch_tensorrt.lib on Windows (Issue #1014) @narendasan<https://github.com/narendasan> there is a def file in //cpp/lib — Reply to this email directly, view it on GitHub<#1014 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AANVFFM2GGMHJEKSDMCKS5DVH4AZRANCNFSM5UWGEY3A>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jonahclarsen]

I tried running bazel build //cpp/lib:torch_tensorrt.lib --compilation_mode opt and got the following (with no .lib file generated):

bazel build //cpp/lib:torch_tensorrt.lib --compilation_mode opt
INFO: Analyzed target //cpp/lib:torch_tensorrt.lib (1 packages loaded, 1 target configured).
INFO: Found 1 target...
INFO: From Linking cpp/lib/torch_tensorrt.lib.dll:
LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
LINK : warning LNK4044: unrecognized option '/Wl,-rpath,lib/'; ignored
Creating library bazel-out/x64_windows-opt/bin/cpp/lib/torch_tensorrt.lib.if.lib and object bazel-out/x64_windows-opt/bin/cpp/lib/torch_tensorrt.lib.if.exp
Target //cpp/lib:torch_tensorrt.lib up-to-date:
bazel-bin/cpp/lib/torch_tensorrt.lib.dll
INFO: Elapsed time: 1.062s, Critical Path: 0.78s
INFO: 9 processes: 8 internal, 1 local.
INFO: Build completed successfully, 9 total actions

I realized it was creating a .lib.dll file, so I changed linkstatic = to False for the .lib target. This resulted in this error:

bazel build //cpp/lib:torch_tensorrt.lib --compilation_mode opt
INFO: Analyzed target //cpp/lib:torch_tensorrt.lib (1 packages loaded, 1 target configured).
INFO: Found 1 target...
ERROR: C:/users/jonah/downloads/torch-tensorrt-1.0.0/core/util/BUILD:72:11: Linking core/util/trt_util_09159dd4ae.dll failed: (Exit 1120): link.exe failed: error executing command C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/link.exe @bazel-out/x64_windows-opt/bin/core/util/trt_util_09159dd4ae.dll-2.params
LINK : warning LNK4044: unrecognized option '/Wl,-rpath,lib/'; ignored
Creating library bazel-out/x64_windows-opt/bin/core/util/trt_util.if.lib and object bazel-out/x64_windows-opt/bin/core/util/trt_util.if.exp
trt_util.obj : error LNK2019: unresolved external symbol "public: void __cdecl torch_tensorrt::core::util::logging::TorchTRTLogger::log(enum torch_tensorrt::core::util::logging::LogLevel,class std::basic_string<char,struct std::char_traits,class std::allocator >)" (?log@TorchTRTLogger@logging@util@core@torch_tensorrt@@QEAAXW4LogLevel@2345@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@@z) referenced in function "class nvinfer1::Dims32 __cdecl torch_tensorrt::core::util::toDimsPad(class c10::ArrayRef<__int64>,unsigned __int64)" (?toDimsPad@util@core@torch_tensorrt@@ya?AVDims32@nvinfer1@@v?$ArrayRef@_J@c10@@_K@Z)
trt_util.obj : error LNK2019: unresolved external symbol "class torch_tensorrt::core::util::logging::TorchTRTLogger & __cdecl torch_tensorrt::core::util::logging::get_logger(void)" (?get_logger@logging@util@core@torch_tensorrt@@YAAEAVTorchTRTLogger@1234@XZ) referenced in function "class nvinfer1::Dims32 __cdecl torch_tensorrt::core::util::toDimsPad(class c10::ArrayRef<__int64>,unsigned __int64)" (?toDimsPad@util@core@torch_tensorrt@@ya?AVDims32@nvinfer1@@v?$ArrayRef@_J@c10@@_K@Z)
trt_util.obj : error LNK2019: unresolved external symbol "public: __cdecl torch_tensorrt::Error::Error(char const *,unsigned int,class std::basic_string<char,struct std::char_traits,class std::allocator > const &,void const *)" (??0Error@torch_tensorrt@@qeaa@PEBDIAEBV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@PeBx@Z) referenced in function "enum nvinfer1::DataType __cdecl torch_tensorrt::core::util::ScalarTypeToTRTDataType(enum c10::ScalarType)" (?ScalarTypeToTRTDataType@util@core@torch_tensorrt@@ya?AW4DataType@nvinfer1@@W4ScalarType@c10@@@z)
bazel-out\x64_windows-opt\bin\core\util\trt_util_09159dd4ae.dll : fatal error LNK1120: 3 unresolved externals
Target //cpp/lib:torch_tensorrt.lib failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: C:/users/jonah/downloads/torch-tensorrt-1.0.0/cpp/lib/BUILD:44:10 Linking cpp/lib/torch_tensorrt.lib.dll failed: (Exit 1120): link.exe failed: error executing command C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/link.exe @bazel-out/x64_windows-opt/bin/core/util/trt_util_09159dd4ae.dll-2.params
INFO: Elapsed time: 0.346s, Critical Path: 0.08s
INFO: 8 processes: 8 internal.
FAILED: Build did NOT complete successfully

[jonahclarsen]

Yes this is //cpp/lib. Just tried clearing the cache, ran mostly til the end then gave the same error but of course ending this time in:

INFO: Elapsed time: 59.844s, Critical Path: 17.10s
INFO: 762 processes: 677 internal, 85 local.
FAILED: Build did NOT complete successfully

[jonahclarsen]

I also modified line 11 of macros.h to #define TORCHTRT_API __attribute__((__visibility__("default"))) __declspec(dllexport) and was able to successfully build the .dll with that as before, the .dll is the same size though so I'm not sure if it made any difference.

[narendasan]

Should it be all one line or should it be it’s own MVSC case for TORCHTRT_API?

…

________________________________ From: Jonah ***@***.***> Sent: Sunday, May 1, 2022 3:53:52 PM To: NVIDIA/Torch-TensorRT ***@***.***> Cc: Naren Dasan ***@***.***>; Mention ***@***.***> Subject: Re: [NVIDIA/Torch-TensorRT] ❓ [Question] Building torch_tensorrt.lib on Windows (Issue #1014) I also modified line 11 of macros.h to #define TORCHTRT_API __attribute__((__visibility__("default"))) __declspec(dllexport) and was able to successfully build the .dll with that as before, the .dll is the same size though so I'm not sure if it made any difference. — Reply to this email directly, view it on GitHub<#1014 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AANVFFIZOODOEXKWTRFV7ZLVH4DQBANCNFSM5UWGEY3A>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jonahclarsen]

I'm not sure what you mean, if I define TORCHTRT_API again then won't it just replace the existing attribute definition?

[jonahclarsen]

I tried it on a new line as so:

#define TORCHTRT_API attribute((visibility("default")))
#define TORCHTRT_API __declspec(dllexport)

And was again able to build the .dll successfully (it was the same size as before), but still not the .lib file.

[narendasan]

I think something more along the lines of this https://ms-iot.github.io/ROSOnWindows/Porting/SymbolVisibility.html

Since I believe the __attribute__ flags are mostly for GCC.