Don't get memcpy, memmove, memcmp, memset, and strlen from msvcrt.dll #29130
Description
The memcpy, memmove, memcmp, strlen, and memset symbols in msvcrt are just shims for implementations in ntdll.dll, which have existed since Windows 3.1.
If possible, we should link against these directly, rather than going through msvcrt. This would have the effect of eliminating all of our dependencies on msvcrt except for math functions and the entry point.
cc @retep998
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Participants
Activity
retep998 commentedon Oct 18, 2015
These functions are not shims as I have just disassembled and verified this. Also
ntdll.dllis an explicitly unstable API that should not be relied on directly so I don't recommend linking to it directly.alexchandel commentedon Oct 18, 2015
@retep998 At least on very old systems, msvcrt.dll did call into
ntdll.dllfor them. That's rather surprising that they would be called unstable, being there unchanged for 25 years. Can you find something on MSDN marking these symbols as unstable?retep998 commentedon Oct 18, 2015
@alexchandel
ntdll.dllis the interface to the NT kernel. The NT kernel is free to add or remove functions with each version of NT. Just because something hasn't changed does not mean Microsoft provides any sort of promise of stability.ntdll.dllhas functions likememcpyand friends because it is providing them for other system libraries. It's not intended to be directly used by user-mode applications.alexchandel commentedon Oct 18, 2015
@retep998 I know it has all the syscalls, and that their syscalls are unstable, but I can't find any documentation at all on what symbols are in the current version. Which is uncharacteristic of MS.
But that's too bad if we can't, because unless we provide our own implementations of them (alongside an allocator like jemalloc and a math lib like openlibm), we're still tied to MS's CRT>
retep998 commentedon Oct 18, 2015
@alexchandel We don't rely on the CRT for the allocator, even without jemalloc. Replacing the
mathmemory functions with our own optimized versions should be quite doable with the new simd stuff that @huonw is working on.huonw commentedon Oct 18, 2015
(I don't think implementing our own libm-replacement is a great idea---using a preexisting one like openlibm seems better for various reasons---and, even then, SIMD isn't so useful for scalar functions like those in libm.)
abonander commentedon Oct 18, 2015
@huonw Can you elaborate on a few of these reasons? From what I understand, memcpy, memmove and memset have relatively trivial implementations. I realize there's probably loads of platform- and architecture-specific optimizations in the big libs but can't we still get reasonably fast with our own implementation? If it eliminates a problematic dependency I'm all for it. At the very least, it could be a viable alternative.
alexchandel commentedon Oct 18, 2015
@cybergeek94 I think he meant implementing math functions. @huonw you're working on SIMD stuff for memory?
huonw commentedon Oct 18, 2015
No, I'm not. I was replying to @retep998's original comment which had a typo: it said "math" instead of "memory".
In any case, rlibc implements the memory functions Rust needs.
alexchandel commentedon Oct 21, 2015
@huonw how does one have rustc link against rlibc instead of msvcrt?
Combined with openlibm and an entry point, that would simplify a cross-compile to windows to
LIB=$HOME/i686-pc-windows-msvc-w10/lib rustc --target i686-pc-windows-msvc test.rs, since LLD plays nicely now (#29126)briansmith commentedon Jan 31, 2016
Why not just remove all uses of
memcpy,memmove,memcmp,strlen, andmemsetfrom the Rust code completely, writing them all in Rust? Not only would that be better than relying on an unstable Windows API, but also that effort would help other efforts for libc-less Rust.huonw commentedon Jan 31, 2016
LLVM inserts calls to (some of) those functions during optimisation passes.
briansmith commentedon Jan 31, 2016
Why not just turn off those optimizations in LLVM? That would be useful for some users of
#![no_std]too.retep998 commentedon Jan 31, 2016
Because then you'd be harming performance for very little gain.
12 remaining items
DemiMarie commentedon Jul 10, 2016
What about ntdll.dll? It is on all Windows systems, and provides such
functions.
Another option is to use one from any of the permissively-licensed libc
implementations. I did not say that we should write our own — musl libc
should have one, as do the BSDs. That is assuming of course that this is
viable — it very well might not be.
On Jul 10, 2016 01:21, "Yonggang Luo" notifications@github.com wrote:
alexchandel commentedon Jul 12, 2016
Currently the options are:
memcpy/etc symbols inntdll.dll, which are undocumented and may go away (but probably won't). C code statically linked with Rust must use a different method.memcpy/etc symbols inmsvcrt.dll, which are provided for compatibility with old Windows programs and won't go away ever. The VC6 and MinGW toolchains can compile C code against this as well. (Note: Rust seems now to depend on symbols added after VC6)memcpy/etc symbols inmsvcrXYZ.dll, which won't go away but aren't always present. This forces Rust programs to either ship the VS Redistributable or to depend on a Windows update. The VC7-VC13 toolchains can compile C code against this as well.memcpy/etc symbols inucrtbase.dll, which won't go away but aren't always present. This forces Rust programs to either ship the UCRT Redistributable or to depend on a Windows update. The VC14 toolchain can compile C code against this as well, which must then ship the VCRuntime redistributable.memcpy/etc assembly from an open-source C library like MinGW (or ask MS to open-source the UCRT), and compile that into Rust programs. C code statically linked with Rust must use a different method.memcpy/etc code and compile that into Rust programs. C code statically linked with Rust must use a different method.Per #30027 none of these truly requires Rust to use a toolchain. And Rust code and C code statically linked with Rust can use different methods.
retep998 commentedon Jul 12, 2016
@alexchandel Don't forget that we could just statically link the CRT which avoids the dependence on redistributables. rust-lang/libc#290
briansmith commentedon Oct 18, 2016
I would prefer for Rust to use the same implementation that the C compiler would use, so that when I link C and Rust code together, they are using a common implementation.
alexchandel commentedon Dec 13, 2016
@retep998 When you statically link against the UCRT in MSVC 14+, does the Rust-only executable have any dependence on the new
VCRUNTIME*.DLL(or the static version,libvcruntime.lib)? It's still required to link bothlibucrt.libandlibcmt.lib, right?And on a parallel note, if C object files are linked into this Rust binary as well, does linking
libvcruntime.libprovide the static equivalent of linkingVCRUNTIME*.DLL?retep998 commentedon Dec 13, 2016
By linking to
libcmt.libyou also link to bothlibucrt.libandlibvcruntime.liband end up with all the CRT stuff statically linked. No vc runtime dlls, no crt dlls. Mixing static and dll versions of the ucrt and vcruntime is not supported.lygstate commentedon Dec 13, 2016
I think we only need static link to libvcruntime.lib for memmove things
retep998 commentedon Dec 14, 2016
@lygstate You already can statically link to the CRT. #37545 landed recently although it is still unstable.
sanmai-NL commentedon Aug 5, 2017
@alexchandel: reviewing your last exchange with @retep998, I would like to ask whether you still wish to keep this issue open?
retep998 commentedon Sep 13, 2019
Given that Rust provides the ability to statically link the CRT, and there are plans to provide a pure Rust Windows target in the future when #58713 is implemented, closing this issue.