Skip to content

Commit 001d8b0

Browse files
HyukjinKwondongjoon-hyun
authored andcommitted
[SPARK-37554][BUILD] Add PyArrow, pandas and plotly to release Docker image dependencies
### What changes were proposed in this pull request? This PR proposes to add plotly, pyarrow and pandas dependencies for generating the API documentation for pandas API on Spark. The versions of `pandas==1.1.5 pyarrow==3.0.0 plotly==5.4.0` are matched with the current versions being used in branch-3.2 at Python 3.6. ### Why are the changes needed? Currently, the function references for pandas API on Spark are all missing: https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/series.html due to missing dependencies when building the docs. ### Does this PR introduce _any_ user-facing change? Yes, the broken links of documentation at https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/series.html will all be recovered. ### How was this patch tested? To be honest, it has not been tested. I don't have the nerve to run Docker releasing script for the sake of testing so I defer to the next release manager. The combinations of the dependency versions are being tested in GitHub Actions at `branch-3.2`. Closes #34813 from HyukjinKwon/SPARK-37554. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]> (cherry picked from commit 03750c0) Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 9dd4c07 commit 001d8b0

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

dev/create-release/spark-rm/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ ARG APT_INSTALL="apt-get install --no-install-recommends -y"
4242
# We should use the latest Sphinx version once this is fixed.
4343
# TODO(SPARK-35375): Jinja2 3.0.0+ causes error when building with Sphinx.
4444
# See also https://issues.apache.org/jira/browse/SPARK-35375.
45-
ARG PIP_PKGS="sphinx==3.0.4 mkdocs==1.1.2 numpy==1.19.4 pydata_sphinx_theme==0.4.1 ipython==7.19.0 nbsphinx==0.8.0 numpydoc==1.1.0 jinja2==2.11.3 twine==3.4.1 sphinx-plotly-directive==0.1.3"
45+
ARG PIP_PKGS="sphinx==3.0.4 mkdocs==1.1.2 numpy==1.19.4 pydata_sphinx_theme==0.4.1 ipython==7.19.0 nbsphinx==0.8.0 numpydoc==1.1.0 jinja2==2.11.3 twine==3.4.1 sphinx-plotly-directive==0.1.3 pandas==1.1.5 pyarrow==3.0.0 plotly==5.4.0"
4646
ARG GEM_PKGS="bundler:2.2.9"
4747

4848
# Install extra needed repos and refresh.

0 commit comments

Comments
 (0)