Windows won't compile without threads

[khwilliamson]

Sample fail report
http://perl.develop-help.com/raw/?id=245942

[khwilliamson]

@greerga anything to add?

[khwilliamson]

@xenu had looked at this some

[gannett-ggreer]

You linked to one of the builds prior to the mbrtowc failure; did you mean one that was failing non-threaded like http://perl.develop-help.com/raw/?id=248349 ?

[khwilliamson]

On 3/2/20 7:12 PM, George Greer wrote: You linked to one of the builds prior to the |mbrtowc| failure; did you mean one that was failing non-threaded like http://perl.develop-help.com/raw/?id=248349 ?

This bug predates the mbrtowc changes, so I deliberately linked to a report that doesn't have them, which would be a distraction from the underlying cause

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17601?email_source=notifications&email_token=AAA2DH2O5HXJOTNJ5AG2QFLRFRRRXA5CNFSM4K7JMRU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENRZR4I#issuecomment-593729777>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2DHY7MJFCNIVADMUVPOLRFRRRXANCNFSM4K7JMRUQ>.

[gannett-ggreer]

Ah ok, you must be referring to these messages.

# Test.obj : error LNK2001: unresolved external symbol PL_curpad
# Test.obj : error LNK2001: unresolved external symbol PL_op
# Test.obj : error LNK2001: unresolved external symbol PL_stack_base
# Test.obj : error LNK2001: unresolved external symbol PL_tainted
# Test.obj : error LNK2001: unresolved external symbol PL_markstack_ptr
# Test.obj : error LNK2001: unresolved external symbol PL_stack_sp
# blib\arch\auto\XS\Test\Test.dll : fatal error LNK1120: 6 unresolved externals
# NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX64\x64\link.EXE"' : return code '0x460'
# Other.obj : error LNK2001: unresolved external symbol PL_curpad
# Other.obj : error LNK2001: unresolved external symbol PL_op
# Other.obj : error LNK2001: unresolved external symbol PL_stack_base
# Other.obj : error LNK2001: unresolved external symbol PL_tainted
# Other.obj : error LNK2001: unresolved external symbol PL_markstack_ptr
# Other.obj : error LNK2001: unresolved external symbol PL_stack_sp
# blib\arch\auto\XS\Other\Other.dll : fatal error LNK1120: 6 unresolved externals
[2020-01-24 19:04:57Z] perlio.obj : warning LNK4197: export 'PerlIO_perlio' specified multiple times; using first specification
[2020-01-24 19:04:57Z] perlio.obj : warning LNK4197: export 'PerlIO_pending' specified multiple times; using first specification
[2020-01-24 19:04:57Z] c:\perl\smoke\perl\mswin32-win10-blead\build\ext\POSIX\POSIX.xs(1090) : warning C4715: 'my_rint': not all control paths return a value
# Test.obj : error LNK2001: unresolved external symbol PL_markstack
# Test.obj : error LNK2001: unresolved external symbol PL_debug
# blib\arch\auto\XS\Test\Test.dll : fatal error LNK1120: 7 unresolved externals
# Other.obj : error LNK2001: unresolved external symbol PL_markstack
# Other.obj : error LNK2001: unresolved external symbol PL_debug
# blib\arch\auto\XS\Other\Other.dll : fatal error LNK1120: 7 unresolved externals

By "won't compile without threads" I thought initially you were referring to the Perl binary itself.

[tonycoz]

This is open upstream as https://rt.cpan.org/Ticket/Display.html?id=115321 (since 2016)

[Leont]

This is open upstream as https://rt.cpan.org/Ticket/Display.html?id=115321 (since 2016)

It rather looks like a problem in blead. On an unthreaded perl build on Windows the interpreter variables need to added to the export list (because they are all global symbols). The error message is suggesting that this isn't working. If the problem was MakeMaker not importing correctly, it would be a much broader group of symbols.

[tonycoz]

I've looked further into this.

The symbols are being exported, but under the conditions in the failing tests, they aren't being properly imported.

For the failing tests, the Makefile is being generated with LINKTYPE=static in Makefile.PL, but then built with LINKTYPE=dynamic.

The LINKTYPE=static Makefile adds -DPERLDLL to the definition of CCFLAGS, without that value depending on the LINKTYPE value, so LINKTYPE=dynamic doesn't override it.

-DPERLDLL controls whether symbols, including data, is marked with __declspec(dlliexport) or __declspec(dllimport) and in the failing cases this means the symbols are being marked for export, which is ignored since the symbols aren't in the image.

Due to the way data is imported on Win32 this means that the symbols then aren't found.

The only fix I can see is for the generated Makefile to make whether CCFLAGS includes -DPERLDLL depend on the LINKTYPE parameter supplied to $(MAKE).

(a note: if the symbols weren't being exported every other test that loads an XS module would also fail.)

[Leont]

-DPERLDLL controls whether symbols, including data, is marked with __declspec(dlliexport) or __declspec(dllimport) and in the failing cases this means the symbols are being marked for export, which is ignored since the symbols aren't in the image.

That is most surprising to me. I'm not sure I understand what the hell makedef.pl is doing with those same symbols then. Is that list only being used for the exporting but not for the importing?

And I still don't quite understand why this only seems to break these symbols.

[tonycoz]

The problem is the importing, not the exporting.

(Code symbols are imported the same way, but it has a mitigation.)

On Win32 the code generated to access an imported symbol is different to the code generated to access a local symbol.

The code you normally get to say fetch a pointer from PL_stack_base is something like:

    mov (PL_stack_base), %eax  ; just fetch it directly

and the linker won't try to resolve that symbol from imports.

On Win32, to reduce the number of relocations, instead only a single relocation is done for each symbol, so the code becomes:

   mov (__imp__PL_stack_base), %eax ; fetch a pointer to PL_stack_base
   mov (%eax), %eax   ; fetch PL_stack_base itself

Something similar is done for code, except that code that directly calls the function can be satisfied by a thunk that either indirectly calls through the pointer, or where the relocation is part of the thunk. That can't work for data.

As to why only those symbols - the XS modules tested are trivial, those are the only data symbols they access.

(It's been a very long time since I looked at this in detail, so I might have some details not quite right.)

[Leont]

For the failing tests, the Makefile is being generated with LINKTYPE=static in Makefile.PL, but then built with LINKTYPE=dynamic.

The problem here is that some of the test-cases have a subdir that is set to LINKTYPE=static in the subdir-Makefile.PL, but then during make the LINKTYPE=dynamic is passed through.

The only fix I can see is for the generated Makefile to make whether CCFLAGS includes -DPERLDLL depend on the LINKTYPE parameter supplied to $(MAKE).

I'm rather wondering why we pass through LINKTYPE in the first place, I don't think I've ever seen a non-static subdir.