`test_strftime_y2k` fails on embedded Linux #123681

picnixz · 2024-09-04T16:30:33Z

Bug report

Bug description:

This is an issue to track the progress of fixing the JIT. Branches that trigger the embedded Linux tests fail on test_strftime_y2k, e.g. https://github.com/python/cpython/actions/runs/10655985897/job/29534232332?pr=123546. As a side-effect this causes users getting notification that their JIT workflow failed.

Both the C and F specifiers are failing for this specific test:

test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) ... 
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) ... 
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) ... 
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) ... 
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL

A solution is to add this test to the ignored tests or to fix the detection algorithm.

cc @blhsing @serhiy-storchaka

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs

Related and proposed PR: gh-128106

The text was updated successfully, but these errors were encountered:

serhiy-storchaka · 2024-09-04T16:52:21Z

It seems that only tests for the C implementation fail.

We can try to move the detection from the compile time to the run time, like for Python implementation. But it is a puzzle why it does not work here.

picnixz · 2024-09-04T16:57:40Z

(Forget my comment that I deleted, I looked at the wrong commit).

I think we can improve the configure.ac detection by including the weird cases. Or maybe the test is broken because of this:

if _time.strftime('%F', (1900, 1, 1, 0, 0, 0, 0, 1, 0)) == '1900-01-01':
    specifiers += 'FC'

(Those are the two failing specifiers)

brandtbucher · 2024-09-06T19:42:50Z

This particular test failure could be a bit weird, since this CI job involves cross-compiling Python and then running the test suite under emulation. Not sure if it's a "real" issue, or if we should just skip the test (like we do for others).

picnixz · 2024-09-07T11:55:25Z

since this CI job involves cross-compiling Python and then running the test suite under emulation

Does it mean that the configure tests might not be correct (or inconsistent because they were done using cross-compilation?) If so, we can just outright ignore the test I guess?

serhiy-storchaka · 2024-09-07T12:42:53Z

If these configure tests are not correct, then other configure tests can be not correct too.

picnixz · 2024-09-08T18:20:24Z

Ha! I knew I should have looked more in detail the logs:

On aarch64-unknown-linux-gnu (failing tests), we have two different configure:

https://github.com/python/cpython/commit/e565b87d9dafe89bf262e3ab46edb9c9c8618b27/checks/29534232332/logs

2024-09-01T17:33:50.4537429Z checking whether year with century should be normalized for strftime... yes
2024-09-01T17:33:50.4888417Z checking whether C99-specific strftime specifiers are supported... yes

2024-09-01T17:35:47.4696152Z checking whether year with century should be normalized for strftime... yes
2024-09-01T17:35:47.4698164Z checking whether C99-specific strftime specifiers are supported... >> no <<

So it appears we have different ./configure. I don't know which one is actually being used at runtime in the emulated environment. Now, both '%F' and '%C' are C99-specific (https://en.cppreference.com/w/c/chrono/strftime) so the following

if _time.strftime('%F', (1900, 1, 1, 0, 0, 0, 0, 1, 0)) == '1900-01-01':
    specifiers += 'FC'

is perhaps a too broad check. Since specifiers 'F' and 'C' are used, we actually expect to support %F, but this contradicts the fact that "C99-specific strftime specifiers are supported" (at least for the second one which seems the most recent and probably the one being used under the hood). The configure check is:

if (strftime(full_date, sizeof(full_date), "%F", &date) && !strcmp(full_date, "1900-01-01")) {
    return 0;
}

In other words, either strftime(full_date, sizeof(full_date), "%F", &date) fails or full_date != 1900-01-01. So maybe %F does not fail. We can probably debug the tests to check whether it _time.strftime that bugs, or if it is the configure test that bugs. I don't know if it's possible for _time.strftime to support %F (which does not directly call strftime()) but at the same time fail the strftime(full_date, sizeof(full_date), "%F", &date) && !strcmp(full_date, "1900-01-01") test.

I'm tired so what I wrote here might be wrong but I'll try to investigate tomorrow.

blhsing · 2024-09-09T10:15:05Z

2024-09-08T16:49:27.5943515Z checking whether year with century should be normalized for strftime... yes
2024-09-08T16:49:27.6286869Z checking whether C99-specific strftime specifiers are supported... yes

2024-09-08T16:51:33.7240722Z checking whether year with century should be normalized for strftime... yes
2024-09-08T16:51:33.7242911Z checking whether C99-specific strftime specifiers are supported... >> no <<

Ahh how could I have not noticed the second configure in the logs! Was scratching my head over why the detection of C99 support got a yes (in the first configure) but the compiled code ended up not handling C99. It makes total sense now.

The real problem is simply that autoconf does not run the test for the AC_RUN_IFELSE macro at all when cross-compiling, but directly performs action-if-cross-compiling as specified, which I thought should be made a no (to be pessimistic) when it should really be a yes because we should make sure to handle %F and %C ourselves when we are unsure if the target platform can handle them.

picnixz · 2024-09-09T10:18:18Z

Ahh how could I have not noticed the second configure in the logs!

Well, isn't it trivial to spot it among 11985 lines 😄? Actually, I found the duplicate configure in the logs by a CTRL+F and... saw that I had two results.

blhsing · 2024-09-09T10:22:00Z

Ahh how could I have not noticed the second configure in the logs!

Well, isn't it trivial to spot it among 11985 lines 😄? Actually, I found the duplicate configure in the logs by a CTRL+F and... saw that I had two results.

I used Ctrl-F too, but the browser limits the search to only what's loaded in the memory, when the output of the second configure is hidden because it is loaded on demand by an AJAX call only if I scroll to it. I should've used the "Search Logs" text box in the page to perform the search for the keyword on the server side instead.

Thanks again!

picnixz · 2024-09-09T10:25:10Z

I used Ctrl-F too, but the browser limits the search to only what's loaded in the memory, when the output of the second configure is hidden because it is loaded on demand by an AJAX call only if I scroll to it. I should've used the "Search Logs" text box in the page to perform the search for the keyword on the server side instead.

Actually, the trick is to "view raw logs". You'll get the entire logs without buffering (easier to read IMO).

picnixz · 2024-09-27T16:16:05Z

FTR: #124466 was merged as a temporary workaround so we need to keep the issue opened even though the job looks successful (it's just that we skip it).

picnixz · 2024-12-21T18:04:39Z

Closing this one in favor of gh-128104 which has a better description and an alternative PR.

serhiy-storchaka · 2024-12-23T09:59:34Z

Why not test this at runtime instead of compile time, like in the Python implementation?

…at the compile time It is needed to support cross-compiling. Remove macros Py_NORMALIZE_CENTURY and Py_STRFTIME_C99_SUPPORT.

serhiy-storchaka · 2025-01-03T11:32:49Z

We still need to fix a check for Py_NORMALIZE_CENTURY which does not work in cross-compiler. It is only possible at run time.

If we require C99 compliant strftime (it is not on Linux for %C), we need to make the following changes:

Remove the runtime check from the Python implementation.
Update the documentation (it still refers to C89). Document all C99 codes, including the E and O modifiers.

erlend-aasland · 2025-01-03T11:52:20Z

it is not on Linux for %C

%C works swell for me on Debian and Ubuntu. Please specify where it does not work instead of putting out such a broad claim. We also explicitly test the %C specifier in our test suite. If it did not work on Linux, CI would be broken.

serhiy-storchaka · 2025-01-03T12:35:42Z

>>> time.strftime("%C", (99, 1, 1, 0, 0, 0, 0, 1, 0))
'0'

In C99 it should be 2 digits.

picnixz added type-bug An unexpected behavior, bug, or error tests Tests in the Lib/test dir labels Sep 4, 2024

picnixz added this to Date and time issues 🕰️ Sep 4, 2024

picnixz mentioned this issue Sep 4, 2024

datetime.strftime("%Y") is not padding correctly #120713

Closed

picnixz mentioned this issue Sep 8, 2024

gh-124044: protect macros expansions using do-while constructions (WIP) #123842

Closed

6 tasks

blhsing added a commit to blhsing/cpython that referenced this issue Sep 9, 2024

pythongh-123681: handle c99-specific specifiers when cross-compiling

7be70ff

bedevere-app bot mentioned this issue Sep 9, 2024

gh-123681: handle c99-specific specifiers for strftime when cross-compiling #123861

Closed

This was referenced Dec 19, 2024

test_strftime_y2k fails while cross-compiling 3.14a3 for x86_64_v2 and x86_64_v3 on Linux #128104

Closed

gh-128104: Remove Py_STRFTIME_C99_SUPPORT; require C99-compliant strftime #128106

Merged

picnixz closed this as completed Dec 21, 2024

github-project-automation bot moved this to Done in Date and time issues 🕰️ Dec 21, 2024

picnixz closed this as not planned Won't fix, can't repro, duplicate, stale Dec 21, 2024

serhiy-storchaka reopened this Dec 23, 2024

erlend-aasland closed this as completed in #128106 Jan 3, 2025

bedevere-app bot mentioned this issue Jan 3, 2025

gh-123681: Check the strftime() behavior at runtime instead of at the compile time #128444

Open

serhiy-storchaka reopened this Jan 3, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`test_strftime_y2k` fails on embedded Linux #123681

`test_strftime_y2k` fails on embedded Linux #123681

picnixz commented Sep 4, 2024 •

edited by bedevere-app bot

Loading

serhiy-storchaka commented Sep 4, 2024

picnixz commented Sep 4, 2024 •

edited

Loading

brandtbucher commented Sep 6, 2024

picnixz commented Sep 7, 2024 •

edited

Loading

serhiy-storchaka commented Sep 7, 2024

picnixz commented Sep 8, 2024 •

edited

Loading

blhsing commented Sep 9, 2024 •

edited

Loading

picnixz commented Sep 9, 2024

blhsing commented Sep 9, 2024

picnixz commented Sep 9, 2024

picnixz commented Sep 27, 2024

picnixz commented Dec 21, 2024

serhiy-storchaka commented Dec 23, 2024

serhiy-storchaka commented Jan 3, 2025

erlend-aasland commented Jan 3, 2025

serhiy-storchaka commented Jan 3, 2025

test_strftime_y2k fails on embedded Linux #123681

test_strftime_y2k fails on embedded Linux #123681

Comments

picnixz commented Sep 4, 2024 • edited by bedevere-app bot Loading

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

serhiy-storchaka commented Sep 4, 2024

picnixz commented Sep 4, 2024 • edited Loading

brandtbucher commented Sep 6, 2024

picnixz commented Sep 7, 2024 • edited Loading

serhiy-storchaka commented Sep 7, 2024

picnixz commented Sep 8, 2024 • edited Loading

blhsing commented Sep 9, 2024 • edited Loading

picnixz commented Sep 9, 2024

blhsing commented Sep 9, 2024

picnixz commented Sep 9, 2024

picnixz commented Sep 27, 2024

picnixz commented Dec 21, 2024

serhiy-storchaka commented Dec 23, 2024

serhiy-storchaka commented Jan 3, 2025

erlend-aasland commented Jan 3, 2025

serhiy-storchaka commented Jan 3, 2025

`test_strftime_y2k` fails on embedded Linux #123681

`test_strftime_y2k` fails on embedded Linux #123681

picnixz commented Sep 4, 2024 •

edited by bedevere-app bot

Loading

picnixz commented Sep 4, 2024 •

edited

Loading

picnixz commented Sep 7, 2024 •

edited

Loading

picnixz commented Sep 8, 2024 •

edited

Loading

blhsing commented Sep 9, 2024 •

edited

Loading