Multiple Test Failures on Alpine Linux with Threads Enabled #20231
Comments
|
I think debugging this will require a stacktrace. |
|
On Alpine: |
|
It appears that it started triggering errors since commit a7ff7ac A simpler test: Running with Adding some debugging around line 854: When using Using |
|
Debugging further with @khwilliamson and patch provided by @khwilliamson : Running |
khwilliamson
added a commit
to khwilliamson/perl5
that referenced
this issue
Sep 3, 2022
This fixes Perl#20231 LC_ALL is a compendium of the individual locale categories, such as LC_CTYPE, LC_NUMERIC, .... When all categories are in the same locale, it acts just like an individual category. But when the categories are not in the same locale, some means must be used to indicate that. Platforms differ in how they represent this. Alpine uses: a;b;c;d;e;f where each letter is replaced by the correct locale for a given category. Which category is in which position is deterministic, and platform-specific. Other platforms separate by a '/'. And glibc uses a more informative format: LC_CTYPE=a;LC_NUMBERIC=b; ... This has the advantage that it's obvious to the reader what is what, and the order in the string is irrelevant. It might be possible, but painful, for a Configure probe to figure out what the syntax is for the current platform. I chose not to do that. A platform might come along with a novel syntax unanticipated by whatever probe we came up with. Instead, perl uses the glibc format internally, and when it needs to get or set LC_ALL from the system, it loops through each category individually, so that by the time it has done all of them, LC_ALL will have been implicitly handled. The breaking commit a7ff7ac failed to do that.
khwilliamson
added a commit
that referenced
this issue
Sep 7, 2022
This fixes #20231 LC_ALL is a compendium of the individual locale categories, such as LC_CTYPE, LC_NUMERIC, .... When all categories are in the same locale, it acts just like an individual category. But when the categories are not in the same locale, some means must be used to indicate that. Platforms differ in how they represent this. Alpine uses: a;b;c;d;e;f where each letter is replaced by the correct locale for a given category. Which category is in which position is deterministic, and platform-specific. Other platforms separate by a '/'. And glibc uses a more informative format: LC_CTYPE=a;LC_NUMBERIC=b; ... This has the advantage that it's obvious to the reader what is what, and the order in the string is irrelevant. It might be possible, but painful, for a Configure probe to figure out what the syntax is for the current platform. I chose not to do that. A platform might come along with a novel syntax unanticipated by whatever probe we came up with. Instead, perl uses the glibc format internally, and when it needs to get or set LC_ALL from the system, it loops through each category individually, so that by the time it has done all of them, LC_ALL will have been implicitly handled. The breaking commit a7ff7ac failed to do that.
scottchiefbaker
pushed a commit
to scottchiefbaker/perl5
that referenced
this issue
Nov 3, 2022
This fixes Perl#20231 LC_ALL is a compendium of the individual locale categories, such as LC_CTYPE, LC_NUMERIC, .... When all categories are in the same locale, it acts just like an individual category. But when the categories are not in the same locale, some means must be used to indicate that. Platforms differ in how they represent this. Alpine uses: a;b;c;d;e;f where each letter is replaced by the correct locale for a given category. Which category is in which position is deterministic, and platform-specific. Other platforms separate by a '/'. And glibc uses a more informative format: LC_CTYPE=a;LC_NUMBERIC=b; ... This has the advantage that it's obvious to the reader what is what, and the order in the string is irrelevant. It might be possible, but painful, for a Configure probe to figure out what the syntax is for the current platform. I chose not to do that. A platform might come along with a novel syntax unanticipated by whatever probe we came up with. Instead, perl uses the glibc format internally, and when it needs to get or set LC_ALL from the system, it loops through each category individually, so that by the time it has done all of them, LC_ALL will have been implicitly handled. The breaking commit a7ff7ac failed to do that.
khwilliamson
added a commit
that referenced
this issue
May 7, 2023
This reverts commit 9e254b0. Date: Wed Apr 5 12:26:26 2023 -0600 This fixes GH #21040 The reverted commit caused failures in platforms using the musl library, notably Alpine Linux. I came up with a fix for that, which instead broke Windows. In looking at that I realized the original fix is incomplete, and that things are too precarious to try to fix so close to 5.38.0. For example, I spent hours, due to a %p format printing 0 for what turned out to be a non-NULL string pointer. I think it has to do do with the fact that the failing code is in the middle of transitioning between threads, and the printing got confused as a result. The reverted commit was part of a series fixing #20155 and #20231. But the earlier part of the series succeeded in fixing those, without that commit, so reverting it should not cause things to break as a result. This whole issue has to do with locales and threading. Those still don't play well together. I have a series of well over 200 commits that address this situation, for applying in early 5.39. My point is that we are a long way from solving these kinds of issues; and they don't come up that much in the field because they just don't get used. The reverted commit would help if it worked properly, but it's not the only thing wrong by a long shot.
khwilliamson
added a commit
that referenced
this issue
May 8, 2023
This reverts commit 9e254b0. Date: Wed Apr 5 12:26:26 2023 -0600 This fixes GH #21040 The reverted commit caused failures in platforms using the musl library, notably Alpine Linux. I came up with a fix for that, which instead broke Windows. In looking at that I realized the original fix is incomplete, and that things are too precarious to try to fix so close to 5.38.0. For example, I spent hours, due to a %p format printing 0 for what turned out to be a non-NULL string pointer. I think it has to do do with the fact that the failing code is in the middle of transitioning between threads, and the printing got confused as a result. The reverted commit was part of a series fixing #20155 and #20231. But the earlier part of the series succeeded in fixing those, without that commit, so reverting it should not cause things to break as a result. This whole issue has to do with locales and threading. Those still don't play well together. I have a series of well over 200 commits that address this situation, for applying in early 5.39. My point is that we are a long way from solving these kinds of issues; and they don't come up that much in the field because they just don't get used. The reverted commit would help if it worked properly, but it's not the only thing wrong by a long shot.
pjacklam
pushed a commit
to pjacklam/perl5
that referenced
this issue
May 20, 2023
This reverts commit 9e254b0. Date: Wed Apr 5 12:26:26 2023 -0600 This fixes GH Perl#21040 The reverted commit caused failures in platforms using the musl library, notably Alpine Linux. I came up with a fix for that, which instead broke Windows. In looking at that I realized the original fix is incomplete, and that things are too precarious to try to fix so close to 5.38.0. For example, I spent hours, due to a %p format printing 0 for what turned out to be a non-NULL string pointer. I think it has to do do with the fact that the failing code is in the middle of transitioning between threads, and the printing got confused as a result. The reverted commit was part of a series fixing Perl#20155 and Perl#20231. But the earlier part of the series succeeded in fixing those, without that commit, so reverting it should not cause things to break as a result. This whole issue has to do with locales and threading. Those still don't play well together. I have a series of well over 200 commits that address this situation, for applying in early 5.39. My point is that we are a long way from solving these kinds of issues; and they don't come up that much in the field because they just don't get used. The reverted commit would help if it worked properly, but it's not the only thing wrong by a long shot.
pjacklam
pushed a commit
to pjacklam/perl5
that referenced
this issue
May 20, 2023
This reverts commit 9e254b0. Date: Wed Apr 5 12:26:26 2023 -0600 This fixes GH Perl#21040 The reverted commit caused failures in platforms using the musl library, notably Alpine Linux. I came up with a fix for that, which instead broke Windows. In looking at that I realized the original fix is incomplete, and that things are too precarious to try to fix so close to 5.38.0. For example, I spent hours, due to a %p format printing 0 for what turned out to be a non-NULL string pointer. I think it has to do do with the fact that the failing code is in the middle of transitioning between threads, and the printing got confused as a result. The reverted commit was part of a series fixing Perl#20155 and Perl#20231. But the earlier part of the series succeeded in fixing those, without that commit, so reverting it should not cause things to break as a result. This whole issue has to do with locales and threading. Those still don't play well together. I have a series of well over 200 commits that address this situation, for applying in early 5.39. My point is that we are a long way from solving these kinds of issues; and they don't come up that much in the field because they just don't get used. The reverted commit would help if it worked properly, but it's not the only thing wrong by a long shot.
khwilliamson
added a commit
to khwilliamson/perl5
that referenced
this issue
Jul 10, 2023
This reverts commit 9e254b0. Date: Wed Apr 5 12:26:26 2023 -0600 This fixes GH Perl#21040 The reverted commit caused failures in platforms using the musl library, notably Alpine Linux. I came up with a fix for that, which instead broke Windows. In looking at that I realized the original fix is incomplete, and that things are too precarious to try to fix so close to 5.38.0. For example, I spent hours, due to a %p format printing 0 for what turned out to be a non-NULL string pointer. I think it has to do do with the fact that the failing code is in the middle of transitioning between threads, and the printing got confused as a result. The reverted commit was part of a series fixing Perl#20155 and Perl#20231. But the earlier part of the series succeeded in fixing those, without that commit, so reverting it should not cause things to break as a result. This whole issue has to do with locales and threading. Those still don't play well together. I have a series of well over 200 commits that address this situation, for applying in early 5.39. My point is that we are a long way from solving these kinds of issues; and they don't come up that much in the field because they just don't get used. The reverted commit would help if it worked properly, but it's not the only thing wrong by a long shot.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Multiple Test Failures on Alpine Linux with Threads Enabled
Examples:
https://perl5.test-smoke.org/report/5022293
https://perl5.test-smoke.org/logfile/5022293
https://perl5.test-smoke.org/report/5022279
https://perl5.test-smoke.org/logfile/5022279
Appears to have started sometime after f7e7b4d.
Note that Alpine Linux uses musl instead of glibc:
The text was updated successfully, but these errors were encountered: