Skip to content

Commit 6ae97e2

Browse files
YikunHyukjinKwon
authored andcommitted
[SPARK-39611][PYTHON][PS] Fix wrong aliases in __array_ufunc__
### What changes were proposed in this pull request? This PR fix the wrong aliases in `__array_ufunc__` ### Why are the changes needed? When running test with numpy 1.23.0 (current latest), hit a bug: `NotImplementedError: pandas-on-Spark objects currently do not support <ufunc 'divide'>.` In `__array_ufunc__` we first call `maybe_dispatch_ufunc_to_dunder_op` to try dunder methods first, and then we try pyspark API. `maybe_dispatch_ufunc_to_dunder_op` is from pandas code. pandas fix a bug pandas-dev/pandas#44822 (comment) pandas-dev/pandas@206b249 when upgrade to numpy 1.23.0, we need to also sync this. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Current CI passed - The exsiting UT `test_series_datetime` already cover this, I also test it in my local env with 1.23.0 ```shell pip install "numpy==1.23.0" python/run-tests --testnames 'pyspark.pandas.tests.test_series_datetime SeriesDateTimeTest.test_arithmetic_op_exceptions' ``` Closes #37078 from Yikun/SPARK-39611. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]> (cherry picked from commit fb48a14) Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 9adfc3a commit 6ae97e2

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

python/pyspark/pandas/numpy_compat.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,7 @@ def maybe_dispatch_ufunc_to_dunder_op(
157157
"true_divide": "truediv",
158158
"power": "pow",
159159
"remainder": "mod",
160-
"divide": "div",
160+
"divide": "truediv",
161161
"equal": "eq",
162162
"not_equal": "ne",
163163
"less": "lt",

0 commit comments

Comments
 (0)