Skip to content

Commit efb5bb7

Browse files
YikunHyukjinKwon
authored andcommitted
[SPARK-39762][INFRA][PS] Support latest numpy in infra
### What changes were proposed in this pull request? Remove infra numpy<1.23.0 version limit to support numpy 1.23+ (latest) in infra. ### Why are the changes needed? After below two PRs merged: #37117: Fix annotation: `python/pyspark/pandas/frame.py:9970: error: Need type annotation for "raveled_column_labels" [var-annotated]` #37078: Fix wrong aliases in __array_ufunc__: `NotImplementedError: pandas-on-Spark objects currently do not support <ufunc 'divide'>` We can now remove limit on infra file to support numpy > 1.23.0. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed and [numpy 1.23.1](https://github.com/Yikun/spark/runs/7314545823?check_suite_focus=true#step:9:49) installed in CI Closes #37175 from Yikun/patch-24. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent c9e7744 commit efb5bb7

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

dev/infra/Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ RUN $APT_INSTALL software-properties-common git libxml2-dev pkg-config curl wget
3131
RUN update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
3232

3333
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.9
34-
RUN python3.9 -m pip install 'numpy<1.23.0' pyarrow 'pandas<1.4.0' scipy xmlrunner plotly>=4.8 sklearn 'mlflow>=1.0' coverage matplotlib
34+
RUN python3.9 -m pip install numpy pyarrow 'pandas<1.4.0' scipy xmlrunner plotly>=4.8 sklearn 'mlflow>=1.0' coverage matplotlib
3535

3636
RUN add-apt-repository ppa:pypy/ppa
3737
RUN apt update
@@ -44,7 +44,7 @@ RUN mkdir -p /usr/local/pypy/pypy3.7 && \
4444
ln -sf /usr/local/pypy/pypy3.7/bin/pypy /usr/local/bin/pypy3
4545

4646
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3
47-
RUN pypy3 -m pip install 'numpy<1.23.0' 'pandas<1.4.0' scipy coverage matplotlib
47+
RUN pypy3 -m pip install numpy 'pandas<1.4.0' scipy coverage matplotlib
4848

4949
RUN $APT_INSTALL gnupg ca-certificates pandoc
5050
RUN echo 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/' >> /etc/apt/sources.list

0 commit comments

Comments
 (0)