MacPython linux build failing

[TomAugspurger]

There are build failures with the manylinux1 32-bit docker image.

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNPY_NO_DEPRECATED_API=0 -Ipandas/_libs/tslibs -I./pandas/_libs/tslibs -I/opt/python/cp38-cp38/lib/python3.8/site-packages/numpy/core/include -I/opt/python/cp38-cp38/include/python3.8 -c pandas/_libs/tslibs/period.c -o build/temp.linux-x86_64-3.8/pandas/_libs/tslibs/period.o
pandas/_libs/tslibs/period.c: In function ‘__pyx_f_6pandas_5_libs_6tslibs_6period_pqyear’:
pandas/_libs/tslibs/period.c:12464:3: warning: ‘__pyx_v_year’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   return __pyx_r;
   ^
pandas/_libs/tslibs/period.c: In function ‘__pyx_f_6pandas_5_libs_6tslibs_6period_pquarter’:
pandas/_libs/tslibs/period.c:12512:3: warning: ‘__pyx_v_quarter’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   return __pyx_r;


building 'pandas._libs.hashtable' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wl,-strip-all -I/usr/local/include -fPIC -DNPY_NO_DEPRECATED_API=0 -Ipandas/_libs/src/klib -I/opt/python/cp38-cp38/lib/python3.8/site-packages/numpy/core/include -I/opt/python/cp38-cp38/include/python3.8 -c pandas/_libs/hashtable.c -o build/temp.linux-i686-3.8/pandas/_libs/hashtable.o -Werror
pandas/_libs/hashtable.c: In function ‘__pyx_f_6pandas_5_libs_9hashtable_value_count_float64’:
pandas/_libs/hashtable.c:27378:41: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
pandas/_libs/hashtable.c:29050:41: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
         for (__pyx_t_10 = 0; __pyx_t_10 < __pyx_t_9; __pyx_t_10+=1) {
                                         ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_8duplicated_uint64’:
pandas/_libs/hashtable.c:29800:38: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
             __pyx_t_10 = ((__pyx_v_k != __pyx_v_table->n_buckets) != 0);
                                      ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_10ismember_uint64’:
pandas/_libs/hashtable.c:30271:177: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
           *__Pyx_BufPtrStrided1d(__pyx_t_5numpy_uint8_t *, __pyx_pybuffernd_result.rcbuffer->pybuffer.buf, __pyx_t_17, __pyx_pybuffernd_result.diminfo[0].strides) = (__pyx_v_k != __pyx_v_table->n_buckets);
                                                                                                                                                                                 ^
pandas/_libs/hashtable.c: In function ‘__pyx_f_6pandas_5_libs_9hashtable_value_count_object’:
pandas/_libs/hashtable.c:30731:33: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for (__pyx_t_8 = 0; __pyx_t_8 < __pyx_t_7; __pyx_t_8+=1) {
                                 ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_14duplicated_object’:
pandas/_libs/hashtable.c:31389:31: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
       __pyx_t_9 = ((__pyx_v_k != __pyx_v_table->n_buckets) != 0);
                               ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_16ismember_object’:
pandas/_libs/hashtable.c:31825:171: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
     *__Pyx_BufPtrStrided1d(__pyx_t_5numpy_uint8_t *, __pyx_pybuffernd_result.rcbuffer->pybuffer.buf, __pyx_t_16, __pyx_pybuffernd_result.diminfo[0].strides) = (__pyx_v_k != __pyx_v_table->n_buckets);
                                                                                                                                                                           ^
pandas/_libs/hashtable.c: In function ‘__pyx_f_6pandas_5_libs_9hashtable_value_count_int64’:
pandas/_libs/hashtable.c:32258:39: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
         for (__pyx_t_9 = 0; __pyx_t_9 < __pyx_t_8; __pyx_t_9+=1) {
                                       ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_20duplicated_int64’:
pandas/_libs/hashtable.c:33007:38: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
             __pyx_t_10 = ((__pyx_v_k != __pyx_v_table->n_buckets) != 0);
                                      ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_22ismember_int64’:
pandas/_libs/hashtable.c:33478:177: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
           *__Pyx_BufPtrStrided1d(__pyx_t_5numpy_uint8_t *, __pyx_pybuffernd_result.rcbuffer->pybuffer.buf, __pyx_t_17, __pyx_pybuffernd_result.diminfo[0].strides) = (__pyx_v_k != __pyx_v_table->n_buckets);
                                                                                                                                                                                 ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_24mode_float64’:
pandas/_libs/hashtable.c:33800:41: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
         for (__pyx_t_13 = 0; __pyx_t_13 < __pyx_t_12; __pyx_t_13+=1) {
                                         ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_26mode_int64’:
pandas/_libs/hashtable.c:34225:41: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
         for (__pyx_t_13 = 0; __pyx_t_13 < __pyx_t_12; __pyx_t_13+=1) {
                                         ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_28mode_uint64’:
pandas/_libs/hashtable.c:34650:41: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
         for (__pyx_t_13 = 0; __pyx_t_13 < __pyx_t_12; __pyx_t_13+=1) {
                                         ^
pandas/_libs/hashtable.c: In function ‘__pyx_pf_6pandas_5_libs_9hashtable_30mode_object’:
pandas/_libs/hashtable.c:35077:35: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for (__pyx_t_13 = 0; __pyx_t_13 < __pyx_t_12; __pyx_t_13+=1) {
                                   ^
cc1: all warnings being treated as errors

To reproduce

FROM quay.io/pypa/manylinux1_i686:latest

RUN git clone --depth=1 https://github.com/pandas-dev/pandas \
 && cd pandas \
 && /opt/python/cp38-cp38/bin/python -m pip install numpy cython python-dateutil pytz
WORKDIR pandas

RUN /opt/python/cp38-cp38/bin/python setup.py build_ext -i

I've uploaded the generated hashtable.c file here.

---

#34409 fixed the following issues.

https://dev.azure.com/pandas-dev/pandas-wheels/_build/results?buildId=35250&view=logs&j=e9afaa34-1a0f-534e-78fc-fae528ccd915&t=4685e914-c192-594f-c25f-e6cef98b7ba4

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wl,-strip-all -I/usr/local/include -fPIC -DNPY_NO_DEPRECATED_API=0 -Ipandas/_libs/tslibs -I./pandas/_libs/tslibs -I/opt/python/cp38-cp38/lib/python3.8/site-packages/numpy/core/include -I/opt/python/cp38-cp38/include/python3.8 -c pandas/_libs/tslibs/period.c -o build/temp.linux-x86_64-3.8/pandas/_libs/tslibs/period.o -Werror
pandas/_libs/tslibs/period.c: In function ‘__pyx_f_6pandas_5_libs_6tslibs_6period_pqyear’:
pandas/_libs/tslibs/period.c:12479:3: error: ‘__pyx_v_year’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   return __pyx_r;
   ^
pandas/_libs/tslibs/period.c: In function ‘__pyx_f_6pandas_5_libs_6tslibs_6period_pquarter’:
pandas/_libs/tslibs/period.c:12527:3: error: ‘__pyx_v_quarter’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   return __pyx_r;
   ^
cc1: all warnings being treated as errors
warning: pandas/_libs/groupby.pyx:1101:26: Unreachable code

https://dev.azure.com/pandas-dev/pandas-wheels/_build/results?buildId=34768&view=results from May 3rd may be the first one with this warning. https://dev.azure.com/pandas-dev/pandas-wheels/_build/results?buildId=34727&view=results from May 2nd built successfully.

[TomAugspurger]

These are the commits to pandas from the 1st to the 4th

a3477c769b BUG: List indexer on PeriodIndex doesn't coerce strings (#30515) (#30515)
dd84044528 BUG: Series construction with EA dtype and index but no data fails (#33846)
14eda58658 REF: move bits of offsets to liboffsets, de-privatize (#33936)
862db64212 BUG: Fix small memory access errors in ujson objToJSON.c (#33929)
7aa710a4d4 TYP: disallow decorator preserves function signature (#33521)
fd3e5a188f ENH: general concat with ExtensionArrays through find_common_type (#33607)
0bb677e2ef PERF: Datetimelike lookups (#33933)
f76c4c2eef TST: stricter xfails, catch/suppress warnings (#33882)
551c515db7 TST/REF: Move static methods out of consistency classes (#33927)
75a44239b9 TST: tighten xfails in test_strings (#33932)
07402f6c35 PERF: use fastpaths for is_period_dtype checks (#33937)
911e19bf08 [#33770] bug fix to prevent ExtensionArrays from crashing Series.__repr__() (#33771)
29c820fad0 BUG: DTI/TDI intersection result names (#33904)
81093ba821 BLD: bump numpy min version to 1.15.4 (#33729)
b62a3a4f5f CLN: address TODOs (#33886)
be58cd9d8a REF: implement _unbox to de-duplicate unwrapping (#33906)
d149f4109d Create numba helper function for jitting + generating cache key (#33910)
592db7df8d 30999 fix bare pytest raises (#33925)
32dd55cb83 REGR: fix DataFrame reduction with EA columns and numeric_only=True (#33761)
b7f061c3d2 BUG: can't concatenate DataFrame with Series with duplicate keys (#33805)

[TomAugspurger]

@jbrockmendel or @WillAyd do you haev any guesses on this? Looking at tslibs/period.pyx,

@cython.cdivision
cdef int pqyear(int64_t ordinal, int freq):
    cdef:
        int year, quarter
    get_yq(ordinal, freq, &quarter, &year)
    return year

Do you know where year is coming from?

[jbrockmendel]

get_yq calls quarter[0] = DtoQ_yq(unix_date, &af_info, year) and in DtoQ_yq year is set with year[0] = dts.year

I'll look at this more closely after some caffeine

[WillAyd]

Do you know of any reason why the lhs of those assign to array indices? I think more correct to assign to a dereferenced pointer which might get rid of the warning.

…

Sent from my iPhone

On May 12, 2020, at 7:31 AM, jbrockmendel ***@***.***> wrote:  get_yq calls quarter[0] = DtoQ_yq(unix_date, &af_info, year) and in DtoQ_yq year is set with year[0] = dts.year I'll look at this more closely after some caffeine — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[WillAyd]

Nvm that’s a Cython requirement. https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#types So probably fine

…

Sent from my iPhone

On May 12, 2020, at 8:23 AM, William Ayd ***@***.***> wrote: Do you know of any reason why the lhs of those assign to array indices? I think more correct to assign to a dereferenced pointer which might get rid of the warning. Sent from my iPhone >> On May 12, 2020, at 7:31 AM, jbrockmendel ***@***.***> wrote: >> >  > get_yq calls quarter[0] = DtoQ_yq(unix_date, &af_info, year) and in DtoQ_yq year is set with year[0] = dts.year > > I'll look at this more closely after some caffeine > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or unsubscribe.

[hroff-1902]

Why does pqyear() in tslibs/period.pyx has the @cython.cdivision directive applied, while nearest pquarter(), which accepts same parameters, calls get_yq() with same parameters and only returns another value, does not have this directive?

[jbrockmendel]

Why does pqyear() in tslibs/period.pyx has the @cython.cdivision directive applied

Best guess is thats leftover from a time when division was actually done within pqyear

[hroff-1902]

Cannot this cause this problem, I mean?