Merge remote-tracking branch 'origin/main' into more_load_attr

Author: Fidget-Spinner

.github/workflows/build.yml

@@ -235,7 +235,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        openssl_ver: [1.1.1s, 3.0.7]
+        openssl_ver: [1.1.1s, 3.0.7, 3.1.0-beta1]
     env:
       OPENSSL_VER: ${{ matrix.openssl_ver }}
       MULTISSL_DIR: ${{ github.workspace }}/multissl

.github/workflows/doc.yml

@@ -50,18 +50,8 @@ jobs:
       run: make -C Doc/ venv
     - name: 'Check documentation'
       run: make -C Doc/ check
-    - name: 'Upload NEWS'
-      uses: actions/upload-artifact@v3
-      with:
-        name: NEWS
-        path: Doc/build/NEWS
     - name: 'Build HTML documentation'
       run: make -C Doc/ SPHINXOPTS="-q" SPHINXERRORHANDLING="-W --keep-going" html
-    - name: 'Upload docs'
-      uses: actions/upload-artifact@v3
-      with:
-        name: doc-html
-        path: Doc/build/html
 
   # Run "doctest" on HEAD as new syntax doesn't exist in the latest stable release
   doctest:

Doc/c-api/apiabiversion.rst

@@ -58,6 +58,8 @@ See :ref:`stable` for a discussion of API and ABI stability across versions.
    Thus ``3.4.1a2`` is hexversion ``0x030401a2`` and ``3.10.0`` is
    hexversion ``0x030a00f0``.
 
+   Use this for numeric comparisons, e.g. ``#if PY_VERSION_HEX >= ...``.
+
    This version is also available via the symbol :data:`Py_Version`.
 
 .. c:var:: const unsigned long Py_Version

Doc/c-api/arg.rst

@@ -34,24 +34,39 @@ These formats allow accessing an object as a contiguous chunk of memory.
 You don't have to provide raw storage for the returned unicode or bytes
 area.
 
-In general, when a format sets a pointer to a buffer, the buffer is
-managed by the corresponding Python object, and the buffer shares
-the lifetime of this object.  You won't have to release any memory yourself.
-The only exceptions are ``es``, ``es#``, ``et`` and ``et#``.
-
-However, when a :c:type:`Py_buffer` structure gets filled, the underlying
-buffer is locked so that the caller can subsequently use the buffer even
-inside a :c:type:`Py_BEGIN_ALLOW_THREADS` block without the risk of mutable data
-being resized or destroyed.  As a result, **you have to call**
-:c:func:`PyBuffer_Release` after you have finished processing the data (or
-in any early abort case).
-
 Unless otherwise stated, buffers are not NUL-terminated.
 
-Some formats require a read-only :term:`bytes-like object`, and set a
-pointer instead of a buffer structure.  They work by checking that
-the object's :c:member:`PyBufferProcs.bf_releasebuffer` field is ``NULL``,
-which disallows mutable objects such as :class:`bytearray`.
+There are three ways strings and buffers can be converted to C:
+
+*  Formats such as ``y*`` and ``s*`` fill a :c:type:`Py_buffer` structure.
+   This locks the underlying buffer so that the caller can subsequently use
+   the buffer even inside a :c:type:`Py_BEGIN_ALLOW_THREADS`
+   block without the risk of mutable data being resized or destroyed.
+   As a result, **you have to call** :c:func:`PyBuffer_Release` after you have
+   finished processing the data (or in any early abort case).
+
+*  The ``es``, ``es#``, ``et`` and ``et#`` formats allocate the result buffer.
+   **You have to call** :c:func:`PyMem_Free` after you have finished
+   processing the data (or in any early abort case).
+
+*  .. _c-arg-borrowed-buffer:
+
+   Other formats take a :class:`str` or a read-only :term:`bytes-like object`,
+   such as :class:`bytes`, and provide a ``const char *`` pointer to
+   its buffer.
+   In this case the buffer is "borrowed": it is managed by the corresponding
+   Python object, and shares the lifetime of this object.
+   You won't have to release any memory yourself.
+
+   To ensure that the underlying buffer may be safely borrowed, the object's
+   :c:member:`PyBufferProcs.bf_releasebuffer` field must be ``NULL``.
+   This disallows common mutable objects such as :class:`bytearray`,
+   but also some read-only objects such as :class:`memoryview` of
+   :class:`bytes`.
+
+   Besides this ``bf_releasebuffer`` requirement, there is no check to verify
+   whether the input object is immutable (e.g. whether it would honor a request
+   for a writable buffer, or whether another thread can mutate the data).
 
 .. note::
 
@@ -89,7 +104,7 @@ which disallows mutable objects such as :class:`bytearray`.
    Unicode objects are converted to C strings using ``'utf-8'`` encoding.
 
 ``s#`` (:class:`str`, read-only :term:`bytes-like object`) [const char \*, :c:type:`Py_ssize_t`]
-   Like ``s*``, except that it doesn't accept mutable objects.
+   Like ``s*``, except that it provides a :ref:`borrowed buffer <c-arg-borrowed-buffer>`.
    The result is stored into two C variables,
    the first one a pointer to a C string, the second one its length.
    The string may contain embedded null bytes. Unicode objects are converted
@@ -108,8 +123,9 @@ which disallows mutable objects such as :class:`bytearray`.
    pointer is set to ``NULL``.
 
 ``y`` (read-only :term:`bytes-like object`) [const char \*]
-   This format converts a bytes-like object to a C pointer to a character
-   string; it does not accept Unicode objects.  The bytes buffer must not
+   This format converts a bytes-like object to a C pointer to a
+   :ref:`borrowed <c-arg-borrowed-buffer>` character string;
+   it does not accept Unicode objects.  The bytes buffer must not
    contain embedded null bytes; if it does, a :exc:`ValueError`
    exception is raised.
 

Doc/c-api/structures.rst

@@ -228,29 +228,30 @@ Implementing functions and methods
    Structure used to describe a method of an extension type.  This structure has
    four fields:
 
-   +------------------+---------------+-------------------------------+
-   | Field            | C Type        | Meaning                       |
-   +==================+===============+===============================+
-   | :attr:`ml_name`  | const char \* | name of the method            |
-   +------------------+---------------+-------------------------------+
-   | :attr:`ml_meth`  | PyCFunction   | pointer to the C              |
-   |                  |               | implementation                |
-   +------------------+---------------+-------------------------------+
-   | :attr:`ml_flags` | int           | flag bits indicating how the  |
-   |                  |               | call should be constructed    |
-   +------------------+---------------+-------------------------------+
-   | :attr:`ml_doc`   | const char \* | points to the contents of the |
-   |                  |               | docstring                     |
-   +------------------+---------------+-------------------------------+
-
-The :attr:`ml_meth` is a C function pointer.  The functions may be of different
+   .. c:member:: const char* ml_name
+
+      name of the method
+
+   .. c:member:: PyCFunction ml_meth
+
+      pointer to the C implementation
+
+   .. c:member:: int ml_flags
+
+      flags bits indicating how the call should be constructed
+
+   .. c:member:: const char* ml_doc
+
+      points to the contents of the docstring
+
+The :c:member:`ml_meth` is a C function pointer.  The functions may be of different
 types, but they always return :c:expr:`PyObject*`.  If the function is not of
 the :c:type:`PyCFunction`, the compiler will require a cast in the method table.
 Even though :c:type:`PyCFunction` defines the first parameter as
 :c:expr:`PyObject*`, it is common that the method implementation uses the
 specific C type of the *self* object.
 
-The :attr:`ml_flags` field is a bitfield which can include the following flags.
+The :c:member:`ml_flags` field is a bitfield which can include the following flags.
 The individual flags indicate either a calling convention or a binding
 convention.
 

Doc/c-api/typeobj.rst

@@ -147,7 +147,7 @@ Quick Reference
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
    | :c:member:`~PyTypeObject.tp_vectorcall`        | :c:type:`vectorcallfunc`          |                   |   |   |   |   |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
-   | :c:member:`~PyTypeObject.tp_watched`           | char                              |                   |   |   |   |   |
+   | [:c:member:`~PyTypeObject.tp_watched`]         | char                              |                   |   |   |   |   |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
 
 .. [#slots]

Doc/faq/programming.rst

@@ -113,6 +113,8 @@ Yes.  The coding style required for standard library modules is documented as
 Core Language
 =============
 
+.. _faq-unboundlocalerror:
+
 Why am I getting an UnboundLocalError when the variable has a value?
 --------------------------------------------------------------------
 

Doc/library/asyncio-eventloop.rst

@@ -932,7 +932,8 @@ Watching file descriptors
 
 .. method:: loop.remove_reader(fd)
 
-   Stop monitoring the *fd* file descriptor for read availability.
+   Stop monitoring the *fd* file descriptor for read availability. Returns
+   ``True`` if *fd* was previously being monitored for reads.
 
 .. method:: loop.add_writer(fd, callback, *args)
 
@@ -945,7 +946,8 @@ Watching file descriptors
 
 .. method:: loop.remove_writer(fd)
 
-   Stop monitoring the *fd* file descriptor for write availability.
+   Stop monitoring the *fd* file descriptor for write availability. Returns
+   ``True`` if *fd* was previously being monitored for writes.
 
 See also :ref:`Platform Support <asyncio-platform-support>` section
 for some limitations of these methods.

Doc/library/contextvars.rst

@@ -144,6 +144,11 @@ Manual Context Management
    To get a copy of the current context use the
    :func:`~contextvars.copy_context` function.
 
+   Every thread will have a different top-level :class:`~contextvars.Context`
+   object. This means that a :class:`ContextVar` object behaves in a similar
+   fashion to :func:`threading.local()` when values are assigned in different
+   threads.
+
    Context implements the :class:`collections.abc.Mapping` interface.
 
    .. method:: run(callable, *args, **kwargs)

Doc/library/functions.rst

@@ -1733,6 +1733,10 @@ are always available.  They are listed here in alphabetical order.
    .. versionchanged:: 3.8
       The *start* parameter can be specified as a keyword argument.
 
+   .. versionchanged:: 3.12 Summation of floats switched to an algorithm
+      that gives higher accuracy on most builds.
+
+
 .. class:: super()
            super(type, object_or_type=None)
 

Doc/library/imaplib.rst

@@ -187,7 +187,7 @@ IMAP4 Objects
 -------------
 
 All IMAP4rev1 commands are represented by methods of the same name, either
-upper-case or lower-case.
+uppercase or lowercase.
 
 All arguments to commands are converted to strings, except for ``AUTHENTICATE``,
 and the last argument to ``APPEND`` which is passed as an IMAP4 literal.  If

Doc/library/inspect.rst

@@ -343,15 +343,36 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
 
 .. function:: iscoroutinefunction(object)
 
-   Return ``True`` if the object is a :term:`coroutine function`
-   (a function defined with an :keyword:`async def` syntax).
+   Return ``True`` if the object is a :term:`coroutine function` (a function
+   defined with an :keyword:`async def` syntax), a :func:`functools.partial`
+   wrapping a :term:`coroutine function`, or a sync function marked with
+   :func:`markcoroutinefunction`.
 
    .. versionadded:: 3.5
 
    .. versionchanged:: 3.8
       Functions wrapped in :func:`functools.partial` now return ``True`` if the
       wrapped function is a :term:`coroutine function`.
 
+   .. versionchanged:: 3.12
+      Sync functions marked with :func:`markcoroutinefunction` now return
+      ``True``.
+
+
+.. function:: markcoroutinefunction(func)
+
+   Decorator to mark a callable as a :term:`coroutine function` if it would not
+   otherwise be detected by :func:`iscoroutinefunction`.
+
+   This may be of use for sync functions that return a :term:`coroutine`, if
+   the function is passed to an API that requires :func:`iscoroutinefunction`.
+
+   When possible, using an :keyword:`async def` function is preferred. Also
+   acceptable is calling the function and testing the return with
+   :func:`iscoroutine`.
+
+   .. versionadded:: 3.12
+
 
 .. function:: iscoroutine(object)
 

Doc/library/itertools.rst

@@ -834,7 +834,8 @@ which incur interpreter overhead.
        return chain.from_iterable(repeat(tuple(iterable), n))
 
    def dotproduct(vec1, vec2):
-       return sum(map(operator.mul, vec1, vec2))
+       "Compute a sum of products."
+       return sum(starmap(operator.mul, zip(vec1, vec2, strict=True)))
 
    def convolve(signal, kernel):
        # See:  https://betterexplained.com/articles/intuitive-convolution/
@@ -846,7 +847,7 @@ which incur interpreter overhead.
        window = collections.deque([0], maxlen=n) * n
        for x in chain(signal, repeat(0, n-1)):
            window.append(x)
-           yield sum(map(operator.mul, kernel, window))
+           yield dotproduct(kernel, window)
 
    def polynomial_from_roots(roots):
        """Compute a polynomial's coefficients from its roots.

Doc/library/math.rst

@@ -108,12 +108,7 @@ Number-theoretic and representation functions
 .. function:: fsum(iterable)
 
    Return an accurate floating point sum of values in the iterable.  Avoids
-   loss of precision by tracking multiple intermediate partial sums:
-
-        >>> sum([.1, .1, .1, .1, .1, .1, .1, .1, .1, .1])
-        0.9999999999999999
-        >>> fsum([.1, .1, .1, .1, .1, .1, .1, .1, .1, .1])
-        1.0
+   loss of precision by tracking multiple intermediate partial sums.
 
    The algorithm's accuracy depends on IEEE-754 arithmetic guarantees and the
    typical case where the rounding mode is half-even.  On some non-Windows

Doc/library/pathlib.rst

@@ -490,7 +490,7 @@ Pure paths provide the following methods and properties:
       True
 
 
-.. method:: PurePath.is_relative_to(*other)
+.. method:: PurePath.is_relative_to(other)
 
    Return whether or not this path is relative to the *other* path.
 
@@ -502,6 +502,10 @@ Pure paths provide the following methods and properties:
 
    .. versionadded:: 3.9
 
+   .. deprecated-removed:: 3.12 3.14
+
+      Passing additional arguments is deprecated; if supplied, they are joined
+      with *other*.
 
 .. method:: PurePath.is_reserved()
 
@@ -564,7 +568,7 @@ Pure paths provide the following methods and properties:
       True
 
 
-.. method:: PurePath.relative_to(*other, walk_up=False)
+.. method:: PurePath.relative_to(other, walk_up=False)
 
    Compute a version of this path relative to the path represented by
    *other*.  If it's impossible, :exc:`ValueError` is raised::
@@ -581,7 +585,7 @@ Pure paths provide the following methods and properties:
           raise ValueError(error_message.format(str(self), str(formatted)))
       ValueError: '/etc/passwd' is not in the subpath of '/usr' OR one path is relative and the other is absolute.
 
-When *walk_up* is False (the default), the path must start with *other*.
+   When *walk_up* is False (the default), the path must start with *other*.
    When the argument is True, ``..`` entries may be added to form the
    relative path. In all other cases, such as the paths referencing
    different drives, :exc:`ValueError` is raised.::
@@ -605,6 +609,10 @@ When *walk_up* is False (the default), the path must start with *other*.
    .. versionadded:: 3.12
       The *walk_up* argument (old behavior is the same as ``walk_up=False``).
 
+   .. deprecated-removed:: 3.12 3.14
+
+      Passing additional positional arguments is deprecated; if supplied,
+      they are joined with *other*.
 
 .. method:: PurePath.with_name(name)
 

Doc/library/re.rst

@@ -591,10 +591,9 @@ character ``'$'``.
 
 ``\w``
    For Unicode (str) patterns:
-      Matches Unicode word characters; this includes most characters
-      that can be part of a word in any language, as well as numbers and
-      the underscore. If the :const:`ASCII` flag is used, only
-      ``[a-zA-Z0-9_]`` is matched.
+      Matches Unicode word characters; this includes alphanumeric characters (as defined by :meth:`str.isalnum`)
+      as well as the underscore (``_``).
+      If the :const:`ASCII` flag is used, only ``[a-zA-Z0-9_]`` is matched.
 
    For 8-bit (bytes) patterns:
       Matches characters considered alphanumeric in the ASCII character set;

Doc/library/sched.rst

@@ -44,16 +44,22 @@ Example::
    ...     print(time.time())
    ...     s.enter(10, 1, print_time)
    ...     s.enter(5, 2, print_time, argument=('positional',))
+   ...     # despite having higher priority, 'keyword' runs after 'positional' as enter() is relative
    ...     s.enter(5, 1, print_time, kwargs={'a': 'keyword'})
+   ...     s.enterabs(1_650_000_000, 10, print_time, argument=("first enterabs",))
+   ...     s.enterabs(1_650_000_000, 5, print_time, argument=("second enterabs",))
    ...     s.run()
    ...     print(time.time())
    ...
    >>> print_some_times()
-   930343690.257
-   From print_time 930343695.274 positional
-   From print_time 930343695.275 keyword
-   From print_time 930343700.273 default
-   930343700.276
+   1652342830.3640375
+   From print_time 1652342830.3642538 second enterabs
+   From print_time 1652342830.3643398 first enterabs
+   From print_time 1652342835.3694863 positional
+   From print_time 1652342835.3696074 keyword
+   From print_time 1652342840.369612 default
+   1652342840.3697174
+
 
 .. _scheduler-objects: