gh-108767: Replace ctype.h functions with pyctype.h functions

[vstinner]

Replace <ctype.h> locale dependent functions with Python "pyctype.h" locale independent functions:

• Replace isalpha() with Py_ISALPHA().
• Replace isdigit() with Py_ISDIGIT().
• Replace isxdigit() with Py_ISXDIGIT().
• Replace tolower() with Py_TOLOWER().

Leave Modules/_sre/sre.c unchanged, it uses locale dependent functions on purpose.

• Issue: Avoid locale dependent <ctype.h> functions like isascii(), isdigit(), tolower() #108767

[vstinner]

Replace isdigit() with Py_ISDIGIT().
Replace isxdigit() with Py_ISXDIGIT().

On Linux, they don't seem to be locale dependent:

$ man isalpha
(...)

       isdigit()
              checks for a digit (0 through 9).

(...)

       isxdigit()
              checks for hexadecimal digits, that is, one of
              0 1 2 3 4 5 6 7 8 9 a b c d e f A B C D E F.

But I prefer to avoid <ctype.h> functions anyway, just in case.

[vstinner]

cc @serhiy-storchaka

[vstinner]

Merged. Thanks for the review @serhiy-storchaka.

I'm not sure that there is currently a bug being solved here, so in case of doubt, I prefer to not backport the change. This change is more to help me reducing the number of standard header files included by <Python.h>.

[serhiy-storchaka]

I have been thinking about making these changes for a long time, but there were always more urgent matters. So thank you for doing this.

There was a possible pitfall if the argument is EOF or larger than 255. But I checked and found no such cases.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable LTO 3.x has failed when building commit 578ebc5.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/336/builds/3881) and take a look at the build logs.
4. Check if the failure is related to this commit (578ebc5) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/336/builds/3881

Failed tests:

• test.test_concurrent_futures.test_shutdown

Failed subtests:

• test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolSpawnProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

447 tests OK.

10 slowest tests:

• test_gdb: 3 min 37 sec
• test.test_concurrent_futures.test_wait: 1 min 10 sec
• test_signal: 1 min 1 sec
• test.test_multiprocessing_spawn.test_processes: 57.7 sec
• test_socket: 52.2 sec
• test.test_multiprocessing_forkserver.test_processes: 43.7 sec
• test_subprocess: 43.4 sec
• test_io: 40.1 sec
• test_math: 36.8 sec
• test.test_multiprocessing_spawn.test_misc: 35.4 sec

1 test failed:
test.test_concurrent_futures.test_shutdown

14 tests skipped:
test.test_asyncio.test_windows_events
test.test_asyncio.test_windows_utils test_devpoll test_ioctl
test_kqueue test_launcher test_startfile test_tkinter test_ttk
test_winconsoleio test_winreg test_winsound test_wmi
test_zipfile64

1 re-run test:
test.test_concurrent_futures.test_shutdown

Total duration: 3 min 51 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/test/test_concurrent_futures/test_shutdown.py", line 49, in test_interpreter_shutdown
    self.assertFalse(err)
AssertionError: b'Exception in thread Thread-1:\nTraceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/threading.py", line 1059, in _bootstrap_inner\n    self.run()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/concurrent/futures/process.py", line 339, in run\n    self.add_call_item_to_queue()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/concurrent/futures/process.py", line 394, in add_call_item_to_queue\n    self.call_queue.put(_CallItem(work_id,\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/multiprocessing/queues.py", line 94, in put\n    self._start_thread()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/multiprocessing/queues.py", line 177, in _start_thread\n    self._thread.start()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/threading.py", line 978, in start\n    _start_new_thread(self._bootstrap, ())\nRuntimeError: can\'t create new thread at interpreter shutdown\nTraceback (most recent call last):\n  File "<string>", line 1, in <module>\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/multiprocessing/spawn.py", line 122, in spawn_main\n    exitcode = _main(fd, parent_sentinel)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/multiprocessing/spawn.py", line 132, in _main\n    self = reduction.pickle.load(from_parent)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/multiprocessing/synchronize.py", line 115, in __setstate__\n    self._semlock = _multiprocessing.SemLock._rebuild(*state)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n' is not false

[vstinner]

There was a possible pitfall if the argument is EOF or larger than 255. But I checked and found no such cases.

At least, pyctype.h has no undefined behavior, but a well defined behaviors for these values, no?

See PR #108770 which seems to be a more hairy problem than expected!

[serhiy-storchaka]

It also fixes #86842.