Inconsisted behavior of ternary pow() for Python classes and C extension types

[skirpichev]

Bug report

Bug description:

Consider an example:

Python 3.13.1 (tags/v3.13.1:0671451779, Dec  4 2024, 07:55:26) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import example
>>> pow(42, example.xxx(), example.xxx())  # (1)
called power
123
>>> pow(42, example.xxx())
called power
123
>>> class A:
...     def __pow__(self, other, mod=None):
...         print("__pow__")
...         return 123
...     def __rpow__(self, other, mod=None):
...         print("__rpow__")
...         return 321
...         
>>> pow(42, A(), A())  # (2)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    pow(42, A(), A())
    ~~~^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for ** or pow(): 'int', 'A', 'A'
>>> pow(42, A())
__rpow__
321

In presence of the mod argument, C extension type (1) behave differently than the pure-Python analog (2). That looks as a bug. There is no documentation anywhere that explains how any of this is supposed to work so it’s hard to say that an alternative Python implementation like PyPy is doing anything incorrect or not.

Third-party extensions (like gmpy2) already uses (1) to do things like pow(int, gmpy2.mpz, gmpy2.mpz) - work. Unfortunately, it's not possible to implement a pure-Python class like gmpy2.mpz, that able to do this.

example.c

/* example.c */

#define PY_SSIZE_T_CLEAN
#include <Python.h>

PyTypeObject XXX_Type;

static PyObject *
new(PyTypeObject *type, PyObject *args, PyObject *keywds)
{
    return PyObject_New(PyObject, &XXX_Type);
}

static PyObject *
power(PyObject *self, PyObject *other, PyObject *module)
{
    printf("called power\n");
    return PyLong_FromLong(123);
}

static PyNumberMethods xxx_as_number = {
    .nb_power = power,
};

PyTypeObject XXX_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "xxx",
    .tp_new = new,
    .tp_as_number = &xxx_as_number,
};

static struct PyModuleDef ex_module = {
    PyModuleDef_HEAD_INIT,
    "example",
    "Test module.",
    -1,
    NULL,
};

PyMODINIT_FUNC
PyInit_example(void)
{
    PyObject *m = PyModule_Create(&ex_module);
    if (PyModule_AddType(m, &XXX_Type) < 0) {
        return -1;
    }
    return m;
}

PS: This was discussed before in https://discuss.python.org/t/35185.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-130104: Call __rpow__ in ternary pow() if necessary #130251

[skirpichev]

This allows dispatching on the second argument of ternary power:

diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index 1484d9b334..a4902d2680 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -9810,13 +9810,40 @@ slot_nb_power(PyObject *self, PyObject *other, PyObject *modulus)
 {
     if (modulus == Py_None)
         return slot_nb_power_binary(self, other);
-    /* Three-arg power doesn't use __rpow__.  But ternary_op
-       can call this when the second argument's type uses
-       slot_nb_power, so check before calling self.__pow__. */
+    PyObject* stack[3] = {self, other, modulus};
+    int do_other = !Py_IS_TYPE(self, Py_TYPE(other)) &&
+        Py_TYPE(other)->tp_as_number != NULL &&
+        Py_TYPE(other)->tp_as_number->nb_power == slot_nb_power;
     if (Py_TYPE(self)->tp_as_number != NULL &&
         Py_TYPE(self)->tp_as_number->nb_power == slot_nb_power) {
-        PyObject* stack[3] = {self, other, modulus};
-        return vectorcall_method(&_Py_ID(__pow__), stack, 3);
+        PyObject *r;
+        if (do_other && PyType_IsSubtype(Py_TYPE(other), Py_TYPE(self))) {
+            int ok = method_is_overloaded(self, other, &_Py_ID(__rpow__));
+            if (ok < 0) {
+                return NULL;
+            }
+            if (ok) {
+                stack[0] = other;
+                stack[1] = self;
+                r = vectorcall_method(&_Py_ID(__rpow__), stack, 3);
+                if (r != Py_NotImplemented)
+                    return r;
+                Py_DECREF(r);
+                do_other = 0;
+            }
+        }
+        stack[0] = self;
+        stack[1] = other;
+        r = vectorcall_method(&_Py_ID(__pow__), stack, 3);
+        if (r != Py_NotImplemented ||
+            Py_IS_TYPE(other, Py_TYPE(self)))
+            return r;
+        Py_DECREF(r);
+    }
+    if (do_other) {
+        stack[0] = other;
+        stack[1] = self;
+        return vectorcall_method(&_Py_ID(__rpow__), stack, 3);
     }
     Py_RETURN_NOTIMPLEMENTED;
 }

But for third argument, probably there should be a dedicated dunder method.

[serhiy-storchaka]

We do not even need a third-party extension. This issue can be demonstrated by the Python and C implementations of Decimal (see #130230).

>>> from decimal import Decimal
>>> pow(10, Decimal(2), 7)
Decimal('2')
>>> pow(10, 2, Decimal(7))
Decimal('2')
>>> from _pydecimal import Decimal
>>> pow(10, Decimal(2), 7)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    pow(10, Decimal(2), 7)
    ~~~^^^^^^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for ** or pow(): 'int', 'Decimal', 'int'
>>> pow(10, 2, Decimal(7))
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    pow(10, 2, Decimal(7))
    ~~~^^^^^^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for ** or pow(): 'int', 'int', 'Decimal'

[serhiy-storchaka]

See also #122193.