Skip to content

libc++ 18 iostream compatiblity #79933

Not planned
Not planned
@wang-bin

Description

@wang-bin

Since libc++13, iostream classes(for examplebasic_ifstream<char>) explicit instantiations exist in libc++ shared library. When building user code with libc++ <= 17, instantiations are in user code by default unless targeting _LIBCPP_ABI_VERSION >= 2, so we can build with libc++ 17 and run with libc++ < 13(for example android ndk r23, ubuntu 20.04). [In libc++ 18](

#if _LIBCPP_AVAILABILITY_HAS_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1
), _LIBCPP_AVAILABILITY_HAS_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1 is always 1 for linux, so instantiations are not in user code, and produced binary depends on new symbols in libc++ shared library. This can reduce user binary size, but breaks libc++ compatibility, users must redistribute libc++ with their apps or SDKs. But what if an app depends on multiple 3rdparty SDKs built with different libc++ versions? Let the SDK users find out the highest version from these SDKs(how?) and delete others to avoid link error?

My suggestion is

#if defined(_LIBCPP_BUILDING_LIBRARY) || _LIBCPP_ABI_VERSION >= 2
# define _LIBCPP_AVAILABILITY_HAS_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1 1
#else
# define _LIBCPP_AVAILABILITY_HAS_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1 0
#endif

just like what we do for _LIBCPP_ABI_BAD_FUNCTION_CALL_KEY_FUNCTION

Activity

added
libc++libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
on Jan 30, 2024
ldionne

ldionne commented on Jan 30, 2024

@ldionne
Member

If you want to support back-deploying on Linux platforms, you should add availability markup that defines when various symbols were introduced to such platform. We have all the "infrastructure" to support it in place already, it just happens that only Apple seems to take advantage of it: https://github.com/llvm/llvm-project/blob/main/libcxx/include/__availability#L19

ldionne

ldionne commented on Jan 30, 2024

@ldionne
Member

This can reduce user binary size, but breaks libc++ compatibility, users must redistribute libc++ with their apps or SDKs. But what if an app depends on multiple 3rdparty SDKs built with different libc++ versions? Let the SDK users find out the highest version from these SDKs(how?) and delete others to avoid link error?

I don't understand this. If a SDK on Linux is shipping libc++, it needs to be shipping both the headers and the .so (or .a). In that case, the vendor owning the SDK should set up availability markup as described above for their SDK / platform.

wang-bin

wang-bin commented on Jan 30, 2024

@wang-bin
Author

If you want to support back-deploying on Linux platforms, you should add availability markup that defines when various symbols were introduced to such platform. We have all the "infrastructure" to support it in place already, it just happens that only Apple seems to take advantage of it: https://github.com/llvm/llvm-project/blob/main/libcxx/include/__availability#L19

linux is different from apple, there are many distros, for example debian, ubuntu, arch etc. none of them supports availability markup

I don't understand this. If a SDK on Linux is shipping libc++, it needs to be shipping both the headers and the .so (or .a). In that case, the vendor owning the SDK should set up availability markup as described above for their SDK / platform.

No, only SDK header and .so. c++ headers and libs come from toolchain. If c++ headers and .so must be shipped, what about other dependencies, for example libc?

philnik777

philnik777 commented on Jan 30, 2024

@philnik777
Contributor

If you want to support back-deploying on Linux platforms, you should add availability markup that defines when various symbols were introduced to such platform. We have all the "infrastructure" to support it in place already, it just happens that only Apple seems to take advantage of it: https://github.com/llvm/llvm-project/blob/main/libcxx/include/__availability#L19

linux is different from apple, there are many distros, for example debian, ubuntu, arch etc. none of them supports availability markup

Yes, there are lots of distros. We are very aware of that, and we are aware that they don't have availability markups. If you want backwards compatibility support, you have to maintain the markups. Otherwise we have the same policy as libstdc++: you don't get backwards compatibility.

ldionne

ldionne commented on Jan 30, 2024

@ldionne
Member

linux is different from apple, there are many distros, for example debian, ubuntu, arch etc. none of them supports availability markup

The reason why none of them support availability markup is likely that none of these distributions support back-deploying. Like I said, we have the infrastructure in place to support it if desired, I made sure to write it in a vendor-agnostic way when I refactored that area of the codebase a few years ago.

I don't understand this. If a SDK on Linux is shipping libc++, it needs to be shipping both the headers and the .so (or .a). In that case, the vendor owning the SDK should set up availability markup as described above for their SDK / platform.

No, only SDK header and .so. c++ headers and libs come from toolchain. If c++ headers and .so must be shipped, what about other dependencies, for example libc?

I don't understand the vending model you're implying here. Why would you ship the .so in the SDK but the headers in the toolchain? I don't understand how that makes sense. Note that this is what we used to do at Apple and we moved away from it so we now ship everything in the SDK instead. The other option is to ship everything in the toolchain, but I don't understand why it makes sense to ship the .so in the SDK but the headers in the toolchain.

wang-bin

wang-bin commented on Jan 30, 2024

@wang-bin
Author

linux is different from apple, there are many distros, for example debian, ubuntu, arch etc. none of them supports availability markup

The reason why none of them support availability markup is likely that none of these distributions support back-deploying. Like I said, we have the infrastructure in place to support it if desired, I made sure to write it in a vendor-agnostic way when I refactored that area of the codebase a few years ago.

Linux is just an example. The issue is about libc++ comaptibility. I want to build my library using the latest libc++, and run with old libc++. I think that's why _LIBCPP_ABI_BAD_FUNCTION_CALL_KEY_FUNCTION exists. BTW, I solve the problem by explicitly instantiating these classes in my code, but I have to add instantiation code in every shared library and executable, which is not convenient. Before libc++18, I only need to add some build flags, then the result binary is compatible with all libc++ versions.

I don't understand this. If a SDK on Linux is shipping libc++, it needs to be shipping both the headers and the .so (or .a). In that case, the vendor owning the SDK should set up availability markup as described above for their SDK / platform.

No, only SDK header and .so. c++ headers and libs come from toolchain. If c++ headers and .so must be shipped, what about other dependencies, for example libc?

I don't understand the vending model you're implying here. Why would you ship the .so in the SDK but the headers in the toolchain? I don't understand how that makes sense. Note that this is what we used to do at Apple and we moved away from it so we now ship everything in the SDK instead. The other option is to ship everything in the toolchain, but I don't understand why it makes sense to ship the .so in the SDK but the headers in the toolchain.

Because a 3rdparty SDK(for example Qt) doesn't ship libc++/libstdc++, c++ is provided by toolchain. Apple provides everything in the SDK because it's the system SDK, not a 3rdparty SDK. Shipping libc++ in a 3rdparty SDK is not my idea, it's from #71002 (comment)

philnik777

philnik777 commented on Jan 30, 2024

@philnik777
Contributor

linux is different from apple, there are many distros, for example debian, ubuntu, arch etc. none of them supports availability markup

The reason why none of them support availability markup is likely that none of these distributions support back-deploying. Like I said, we have the infrastructure in place to support it if desired, I made sure to write it in a vendor-agnostic way when I refactored that area of the codebase a few years ago.

Linux is just an example. The issue is about libc++ comaptibility. I want to build my library using the latest libc++, and run with old libc++. I think that's why _LIBCPP_ABI_BAD_FUNCTION_CALL_KEY_FUNCTION exists. BTW, I solve the problem by explicitly instantiating these classes in my code, but I have to add instantiation code in every shared library and executable, which is not convenient. Before libc++18, I only need to add some build flags, then the result binary is compatible with all libc++ versions.

We understand what you want to do, and we say that the solution is to add availability markups to libc++. Without markups we're unable to support that, since we'd be unable to extend the dylib. _LIBCPP_ABI_BAD_FUNCTION_CALL_KEY_FUNCTION should also be changed to an availability feature, but that's a bit more complicated. It will most likely happen in the next few releases.

ldionne

ldionne commented on Jan 30, 2024

@ldionne
Member

Closing since I don't think there's more to add here. The path to "fix" this is clear, we have everything in place to support that, but what this issue is requesting (removing the iostreams instantiations by default) is not something we will pursue.

added
wontfixIssue is real, but we can't or won't fix it. Not invalid
on Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    libc++libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.wontfixIssue is real, but we can't or won't fix it. Not invalid

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @ldionne@wang-bin@EugeneZelenko@philnik777

        Issue actions

          libc++ 18 iostream compatiblity · Issue #79933 · llvm/llvm-project