Warnings on the current main branch

[skirpichev]

Bug report

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (main, Aug 28 2023, 13:54:29) [GCC 12.2.0]

A clear and concise description of the bug:

With

$ # Mon Aug 28 13:44:26 MSK 2023
$ wget https://github.com/python/cpython/archive/refs/heads/main.zip
$ unzip -x main.zip
$ cd cpython-main/
$ ./configure -q && make -s

Build is successful, but I see a bunch of warnings:

In function ‘_io_FileIO_close_impl’,
    inlined from ‘_io_FileIO_close’ at ./Modules/_io/clinic/fileio.c.h:34:12:
./Modules/_io/fileio.c:170:9: warning: ‘exc’ may be used uninitialized [-Wmaybe-uninitialized]
  170 |         _PyErr_ChainExceptions1(exc);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Modules/_io/fileio.c: In function ‘_io_FileIO_close’:
./Modules/_io/fileio.c:155:15: note: ‘exc’ was declared here
  155 |     PyObject *exc;
      |               ^~~
Note: Deepfreeze may have added some global objects,
      so run 'make regen-global-objects' if necessary.
In function ‘word_to_string’,
    inlined from ‘coeff_to_string’ at ./Modules/_decimal/libmpdec/io.c:410:13,
    inlined from ‘_mpd_to_string’ at ./Modules/_decimal/libmpdec/io.c:611:18:
./Modules/_decimal/libmpdec/io.c:348:40: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:359:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  359 |     case 15: EXTRACT_DIGIT(s, x, 100000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:348:40: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:360:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  360 |     case 14: EXTRACT_DIGIT(s, x, 10000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:348:40: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:361:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  361 |     case 13: EXTRACT_DIGIT(s, x, 1000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
[...]

N.B. second type of warnings (expansions of EXTRACT_DIGIT) seems to be related to the gcc-12. The first I've seen on the Debian Buster (with gcc-10 per default).

Linked PRs

• gh-108562: fix compiler warnings for bundled libmpdec in the main #114751
• gh-108562: Revert enabling -fstrict-overflow for libmpdec #116302
• gh-129467: turn off false-positive -Wstringop-overflow in word_to_string() #116346

[sobolevn]

On macos it produces a different result:

» ./configure -q && make -s
Python/optimizer.c:388:9: warning: variable 'reserved' set but not used [-Wunused-but-set-variable]
    int reserved = 0;
        ^
1 warning generated.
Note: Deepfreeze may have added some global objects,
      so run 'make regen-global-objects' if necessary.
./Modules/readline.c:1257:21: warning: incompatible function pointer types assigning to 'Function *' (aka 'int (*)(const char *, int)') from 'int (void)' [-Wincompatible-function-pointer-types]
    rl_startup_hook = on_startup_hook;
                    ^ ~~~~~~~~~~~~~~~
./Modules/readline.c:1259:23: warning: incompatible function pointer types assigning to 'Function *' (aka 'int (*)(const char *, int)') from 'int (void)' [-Wincompatible-function-pointer-types]
    rl_pre_input_hook = on_pre_input_hook;
                      ^ ~~~~~~~~~~~~~~~~~
2 warnings generated.
Undefined symbols for architecture arm64:
  "__Py_DECREF_DecRefTotal", referenced from:
      _test_open_code_hook in _testembed.o
      _test_unicode_id_init in _testembed.o
      _Py_DECREF in _testembed.o
  "__Py_NegativeRefcount", referenced from:
      _test_open_code_hook in _testembed.o
      _test_unicode_id_init in _testembed.o
      _Py_DECREF in _testembed.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Programs/_testembed] Error 1

[skirpichev]

@sobolevn, in my case the build was successful. I did update of the bugreport in case it wasn't clear.

[skrah]

The libmpdec warning is a false positive due to -fno-strict-overflow:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113664

libmpdec very much supports strict overflow, so adding -fstrict=overflow to LIBMPDEC_CFLAGS should fix this.

[skirpichev]

@skrah, maybe I miss something, but with --with-strict-overflow I still see warnings:

$ ./configure -q --with-strict-overflow && make -s
In function ‘word_to_string’,
    inlined from ‘coeff_to_string’ at ./Modules/_decimal/libmpdec/io.c:410:13,
    inlined from ‘_mpd_to_string’ at ./Modules/_decimal/libmpdec/io.c:611:18:
./Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
In function ‘word_to_string’,
    inlined from ‘coeff_to_string’ at ./Modules/_decimal/libmpdec/io.c:410:13,
    inlined from ‘_mpd_to_string’ at ./Modules/_decimal/libmpdec/io.c:607:18:
./Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
In function ‘word_to_string’,
    inlined from ‘coeff_to_string’ at ./Modules/_decimal/libmpdec/io.c:410:13,
    inlined from ‘_mpd_to_string’ at ./Modules/_decimal/libmpdec/io.c:501:22:
./Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
  348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
      |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/io.c:356:14: note: in expansion of macro ‘EXTRACT_DIGIT’
  356 |     case 18: EXTRACT_DIGIT(s, x, 100000000000000000ULL, dot);
      |              ^~~~~~~~~~~~~
Checked 110 modules (33 built-in, 76 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)

PS: _io_FileIO_close_impl() from the original report is already fixed.
PPS: gcc version was mentioned above:

$ gcc --version
gcc (Debian 12.2.0-14) 12.2.0

[skrah]

Indeed, that is another false positive (-Warray-bounds this time). It is already disabled in mpdecimal-4.0.0.

As explained in the linked gcc issue, the exact warning emitted depends on what path the optimizer takes.
The key issue in both cases is that gcc does not like the (correct) if (s == dot) check.

If you compile --with-strict-overflow and apply this patch, it should work:

diff --git a/Modules/_decimal/libmpdec/io.c b/Modules/_decimal/libmpdec/io.c
index e7bd6ae..adedde7 100644
--- a/Modules/_decimal/libmpdec/io.c
+++ b/Modules/_decimal/libmpdec/io.c
@@ -46,6 +46,7 @@
 
 
 #if defined(__GNUC__) && !defined(__INTEL_COMPILER) && __GNUC__ >= 7
+  #pragma GCC diagnostic ignored "-Warray-bounds"
   #pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
   #pragma GCC diagnostic ignored "-Wmisleading-indentation"
 #endif

[skirpichev]

Thanks, that part was backported. See full patch: #114751

[encukou]

When this PR was merged, LTO buildbots started failing. See for example: https://buildbot.python.org/all/#/builders/29/builds/5844

[erlend-aasland]

Ok, can you revert #114751, @encukou?

[encukou]

Yes, unless I find a quick fix.

[skirpichev]

#116302 - a partial revert

Maybe in this case we could just ignore -Wstringop-overflow on EXTRACT_DIGIT (not in the whole file!)? From discussion in the gcc issue this looks safe.

[asottile]

deadsnakes 3.13 nightly builds are also failing presumably due to this change as well:

https://github.com/deadsnakes/python3.13-nightly/actions/runs/8129022227/job/22215614577

 x86_64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro      Modules/_decimal/_decimal.o -lm Modules/_decimal/libmpdec/libmpdec.a  -o Modules/_decimal.cpython-313-x86_64-linux-gnu.so
../Modules/_decimal/_decimal.c: In function 'ctx_mpd_issigned':
../Modules/_decimal/libmpdec/mpdecimal.c:322:1: error: inlining failed in call to 'always_inline' 'mpd_issigned': optimization level attribute mismatch
  322 | mpd_issigned(const mpd_t *dec)
      | ^
../Modules/_decimal/_decimal.c:5381:1: note: called from here
 5381 | DecCtx_BoolFunc_NO_CTX(mpd_issigned)
      | ^
../Modules/_decimal/_decimal.c: In function 'ctx_mpd_iszero':
../Modules/_decimal/libmpdec/mpdecimal.c:343:1: error: inlining failed in call to 'always_inline' 'mpd_iszero': optimization level attribute mismatch
  343 | mpd_iszero(const mpd_t *dec)
      | ^
../Modules/_decimal/_decimal.c:5383:1: note: called from here
 5383 | DecCtx_BoolFunc_NO_CTX(mpd_iszero)
      | ^
../Modules/_decimal/_decimal.c: In function 'ctx_mpd_issnan':
../Modules/_decimal/libmpdec/mpdecimal.c:329:1: error: inlining failed in call to 'always_inline' 'mpd_issnan': optimization level attribute mismatch
  329 | mpd_issnan(const mpd_t *dec)
      | ^
../Modules/_decimal/_decimal.c:5382:1: note: called from here
 5382 | DecCtx_BoolFunc_NO_CTX(mpd_issnan)
      | ^
make[2]: *** [/tmp/ccRuYBfF.mk:8: /tmp/ccjop1Fi.ltrans2.ltrans.o] Error 1
make[2]: *** Waiting for unfinished jobs....
../Modules/_decimal/_decimal.c: In function 'dec_mpd_issigned':
../Modules/_decimal/libmpdec/mpdecimal.c:322:1: error: inlining failed in call to 'always_inline' 'mpd_issigned': optimization level attribute mismatch
  322 | mpd_issigned(const mpd_t *dec)
      | ^
../Modules/_decimal/_decimal.c:4367:1: note: called from here
 4367 | Dec_BoolFunc(mpd_issigned)
      | ^
make[2]: *** [/tmp/ccRuYBfF.mk:5: /tmp/ccjop1Fi.ltrans1.ltrans.o] Error 1

[erlend-aasland]

@asottile, can you check if #116302 works for deadsnakes?

[skirpichev]

Alternative fix: #116346

#116302 works for deadsnakes?

I think so: https://github.com/deadsnakes/python3.13-nightly/actions/runs/8153413773

[erlend-aasland]

Thanks for the heads-up, Sergey! Closing as resolved.

[skirpichev]

@erlend-aasland, warnings in the main are present. I think this issue isn't solved.

[skirpichev]

@erlend-aasland, this should be fixed if we update bundled mpdecimal to v4. Does it make sense for you?

[erlend-aasland]

@erlend-aasland, this should be fixed if we update bundled mpdecimal to v4. Does it make sense for you?

Aren't we trying to unbundle mpdecimal? :)

[skirpichev]

Yes, but Debian people removed mpdecimal.

UPD: ah, no. v4.0.0 doesn't include #116346

[vstinner]

@erlend-aasland:

Aren't we trying to unbundle mpdecimal? :)

Right, What's New in Python 3.13 says:

Pending Removal in Python 3.15: The bundled copy of libmpdecimal.