Merge main branch

Author: encukou

.gitattributes

@@ -69,7 +69,7 @@ Doc/library/token-list.inc                          generated
 Include/internal/pycore_ast.h                       generated
 Include/internal/pycore_ast_state.h                 generated
 Include/internal/pycore_opcode.h                    generated
-Include/internal/pycore_runtime_init_generated.h    generated
+Include/internal/pycore_*_generated.h               generated
 Include/opcode.h                                    generated
 Include/token.h                                     generated
 Lib/keyword.py                                      generated
@@ -82,6 +82,7 @@ Parser/parser.c                                     generated
 Parser/token.c                                      generated
 Programs/test_frozenmain.h                          generated
 Python/Python-ast.c                                 generated
+Python/generated_cases.c.h                          generated
 Python/opcode_targets.h                             generated
 Python/stdlib_module_names.h                        generated
 Tools/peg_generator/pegen/grammar_parser.py         generated

.github/CODEOWNERS

@@ -63,7 +63,7 @@ Python/traceback.c            @iritkatriel
 # bytecode.
 **/*import*.c                 @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw
 **/*import*.py                @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw
-**/*importlib/resources/*      @jaraco @warsaw @brettcannon
+**/*importlib/resources/*      @jaraco @warsaw @brettcannon @FFY00
 **/importlib/metadata/*       @jaraco @warsaw
 
 # Dates and times
@@ -137,8 +137,6 @@ Lib/ast.py                    @isidentical
 
 **/*typing*                   @gvanrossum @Fidget-Spinner @JelleZijlstra @AlexWaygood
 
-**/*asyncore                  @giampaolo
-**/*asynchat                  @giampaolo
 **/*ftplib                    @giampaolo
 **/*shutil                    @giampaolo
 
@@ -148,6 +146,8 @@ Lib/ast.py                    @isidentical
 
 **/*tomllib*                  @encukou
 
+**/*sysconfig*                @FFY00
+
 # macOS
 /Mac/                         @python/macos-team
 **/*osx_support*              @python/macos-team

.github/workflows/build.yml

@@ -176,7 +176,7 @@ jobs:
     needs: check_source
     if: needs.check_source.outputs.run_tests == 'true'
     env:
-      OPENSSL_VER: 1.1.1q
+      OPENSSL_VER: 1.1.1s
       PYTHONSTRICTEXTENSIONBUILD: 1
     steps:
     - uses: actions/checkout@v3
@@ -235,7 +235,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        openssl_ver: [1.1.1q, 3.0.5]
+        openssl_ver: [1.1.1s, 3.0.7]
     env:
       OPENSSL_VER: ${{ matrix.openssl_ver }}
       MULTISSL_DIR: ${{ github.workspace }}/multissl
@@ -282,7 +282,7 @@ jobs:
     needs: check_source
     if: needs.check_source.outputs.run_tests == 'true'
     env:
-      OPENSSL_VER: 1.1.1q
+      OPENSSL_VER: 1.1.1s
       PYTHONSTRICTEXTENSIONBUILD: 1
       ASAN_OPTIONS: detect_leaks=0:allocator_may_return_null=1:handle_segv=0
     steps:

.gitignore

@@ -41,7 +41,6 @@ gmon.out
 .DS_Store
 
 *.exe
-!Lib/distutils/command/*.exe
 
 # Ignore core dumps... but not Tools/msi/core/ or the like.
 core
@@ -58,7 +57,6 @@ Doc/.venv/
 Doc/env/
 Doc/.env/
 Include/pydtrace_probes.h
-Lib/distutils/command/*.pdb
 Lib/lib2to3/*.pickle
 Lib/site-packages/*
 !Lib/site-packages/README.txt

Doc/c-api/call.rst

@@ -93,7 +93,7 @@ This is a pointer to a function with the following signature:
    and they must be unique.
    If there are no keyword arguments, then *kwnames* can instead be *NULL*.
 
-.. c:macro:: PY_VECTORCALL_ARGUMENTS_OFFSET
+.. data:: PY_VECTORCALL_ARGUMENTS_OFFSET
 
    If this flag is set in a vectorcall *nargsf* argument, the callee is allowed
    to temporarily change ``args[-1]``. In other words, *args* points to

Doc/c-api/frame.rst

@@ -79,6 +79,27 @@ See also :ref:`Reflection <reflection>`.
    .. versionadded:: 3.11
 
 
+.. c:function:: PyObject* PyFrame_GetVar(PyFrameObject *frame, PyObject *name)
+
+   Get the variable *name* of *frame*.
+
+   * Return a :term:`strong reference` to the variable value on success.
+   * Raise :exc:`NameError` and return ``NULL`` if the variable does not exist.
+   * Raise an exception and return ``NULL`` on error.
+
+   *name* type must be a :class:`str`.
+
+   .. versionadded:: 3.12
+
+
+.. c:function:: PyObject* PyFrame_GetVarString(PyFrameObject *frame, const char *name)
+
+   Similar to :c:func:`PyFrame_GetVar`, but the variable name is a C string
+   encoded in UTF-8.
+
+   .. versionadded:: 3.12
+
+
 .. c:function:: PyObject* PyFrame_GetLocals(PyFrameObject *frame)
 
    Get the *frame*'s ``f_locals`` attribute (:class:`dict`).

Doc/c-api/memory.rst

@@ -95,6 +95,8 @@ for the I/O buffer escapes completely the Python memory manager.
 Allocator Domains
 =================
 
+.. _allocator-domains:
+
 All allocating functions belong to one of three different "domains" (see also
 :c:type:`PyMemAllocatorDomain`). These domains represent different allocation
 strategies and are optimized for different purposes. The specific details on
@@ -479,6 +481,25 @@ Customize Memory Allocators
    See also :c:member:`PyPreConfig.allocator` and :ref:`Preinitialize Python
    with PyPreConfig <c-preinit>`.
 
+   .. warning::
+
+       :c:func:`PyMem_SetAllocator` does have the following contract:
+
+        * It can be called after :c:func:`Py_PreInitialize` and before
+          :c:func:`Py_InitializeFromConfig` to install a custom memory
+          allocator. There are no restrictions over the installed allocator
+          other than the ones imposed by the domain (for instance, the Raw
+          Domain allows the allocator to be called without the GIL held). See
+          :ref:`the section on allocator domains <allocator-domains>` for more
+          information.
+
+        * If called after Python has finish initializing (after
+          :c:func:`Py_InitializeFromConfig` has been called) the allocator
+          **must** wrap the existing allocator. Substituting the current
+          allocator for some other arbitrary one is **not supported**.
+
+
+
 .. c:function:: void PyMem_SetupDebugHooks(void)
 
    Setup :ref:`debug hooks in the Python memory allocators <pymem-debug-hooks>`

Doc/c-api/refcounting.rst

@@ -7,8 +7,8 @@
 Reference Counting
 ******************
 
-The macros in this section are used for managing reference counts of Python
-objects.
+The functions and macros in this section are used for managing reference counts
+of Python objects.
 
 
 .. c:function:: Py_ssize_t Py_REFCNT(PyObject *o)
@@ -129,6 +129,11 @@ objects.
    It is a good idea to use this macro whenever decrementing the reference
    count of an object that might be traversed during garbage collection.
 
+   .. versionchanged:: 3.12
+      The macro argument is now only evaluated once. If the argument has side
+      effects, these are no longer duplicated.
+
+
 .. c:function:: void Py_IncRef(PyObject *o)
 
    Increment the reference count for object *o*. A function version of :c:func:`Py_XINCREF`.
@@ -139,3 +144,40 @@ objects.
 
    Decrement the reference count for object *o*. A function version of :c:func:`Py_XDECREF`.
    It can be used for runtime dynamic embedding of Python.
+
+
+.. c:macro:: Py_SETREF(dst, src)
+
+   Macro safely decrementing the `dst` reference count and setting `dst` to
+   `src`.
+
+   As in case of :c:func:`Py_CLEAR`, "the obvious" code can be deadly::
+
+       Py_DECREF(dst);
+       dst = src;
+
+   The safe way is::
+
+        Py_SETREF(dst, src);
+
+   That arranges to set `dst` to `src` _before_ decrementing reference count of
+   *dst* old value, so that any code triggered as a side-effect of `dst`
+   getting torn down no longer believes `dst` points to a valid object.
+
+   .. versionadded:: 3.6
+
+   .. versionchanged:: 3.12
+      The macro arguments are now only evaluated once. If an argument has side
+      effects, these are no longer duplicated.
+
+
+.. c:macro:: Py_XSETREF(dst, src)
+
+   Variant of :c:macro:`Py_SETREF` macro that uses :c:func:`Py_XDECREF` instead
+   of :c:func:`Py_DECREF`.
+
+   .. versionadded:: 3.6
+
+   .. versionchanged:: 3.12
+      The macro arguments are now only evaluated once. If an argument has side
+      effects, these are no longer duplicated.

Doc/c-api/structures.rst

@@ -606,23 +606,25 @@ Defining Getters and Setters
    Structure to define property-like access for a type. See also description of
    the :c:member:`PyTypeObject.tp_getset` slot.
 
-   +-------------+------------------+-----------------------------------+
-   | Field       | C Type           | Meaning                           |
-   +=============+==================+===================================+
-   | name        | const char \*    | attribute name                    |
-   +-------------+------------------+-----------------------------------+
-   | get         | getter           | C function to get the attribute   |
-   +-------------+------------------+-----------------------------------+
-   | set         | setter           | optional C function to set or     |
-   |             |                  | delete the attribute, if omitted  |
-   |             |                  | the attribute is readonly         |
-   +-------------+------------------+-----------------------------------+
-   | doc         | const char \*    | optional docstring                |
-   +-------------+------------------+-----------------------------------+
-   | closure     | void \*          | optional function pointer,        |
-   |             |                  | providing additional data for     |
-   |             |                  | getter and setter                 |
-   +-------------+------------------+-----------------------------------+
+   .. c:member:: const char* PyGetSetDef.name
+
+      attribute name
+
+   .. c:member:: getter PyGetSetDef.get
+
+      C funtion to get the attribute.
+
+   .. c:member:: setter PyGetSetDef.set
+
+      Optional C function to set or delete the attribute, if omitted the attribute is readonly.
+
+   .. c:member:: const char* PyGetSetDef.doc
+
+      optional docstring
+
+   .. c:member:: void* PyGetSetDef.closure
+
+      Optional function pointer, providing additional data for getter and setter.
 
    The ``get`` function takes one :c:expr:`PyObject*` parameter (the
    instance) and a function pointer (the associated ``closure``)::

Doc/c-api/typeobj.rst

@@ -1245,6 +1245,17 @@ and :c:type:`PyType_Type` effectively act as defaults.)
       **Inheritance:**
 
       This flag is not inherited.
+      However, subclasses will not be instantiable unless they provide a
+      non-NULL :c:member:`~PyTypeObject.tp_new` (which is only possible
+      via the C API).
+
+      .. note::
+
+         To disallow instantiating a class directly but allow instantiating
+         its subclasses (e.g. for an :term:`abstract base class`),
+         do not use this flag.
+         Instead, make :c:member:`~PyTypeObject.tp_new` only succeed for
+         subclasses.
 
       .. versionadded:: 3.10
 

Doc/conf.py

@@ -197,7 +197,6 @@
 coverage_ignore_modules = [
     r'[T|t][k|K]',
     r'Tix',
-    r'distutils.*',
 ]
 
 coverage_ignore_functions = [

Doc/faq/programming.rst

@@ -1279,13 +1279,25 @@ Or, you can use an extension that provides a matrix datatype; `NumPy
 <https://numpy.org/>`_ is the best known.
 
 
-How do I apply a method to a sequence of objects?
--------------------------------------------------
+How do I apply a method or function to a sequence of objects?
+-------------------------------------------------------------
 
-Use a list comprehension::
+To call a method or function and accumulate the return values is a list,
+a :term:`list comprehension` is an elegant solution::
 
    result = [obj.method() for obj in mylist]
 
+   result = [function(obj) for obj in mylist]
+
+To just run the method or function without saving the return values,
+a plain :keyword:`for` loop will suffice::
+
+   for obj in mylist:
+       obj.method()
+
+   for obj in mylist:
+       function(obj)
+
 .. _faq-augmented-assignment-tuple-error:
 
 Why does a_tuple[i] += ['item'] raise an exception when the addition works?

Doc/faq/windows.rst

@@ -167,7 +167,7 @@ How can I embed Python into a Windows application?
 
 Embedding the Python interpreter in a Windows app can be summarized as follows:
 
-1. Do _not_ build Python into your .exe file directly.  On Windows, Python must
+1. Do **not** build Python into your .exe file directly.  On Windows, Python must
    be a DLL to handle importing modules that are themselves DLL's.  (This is the
    first key undocumented fact.)  Instead, link to :file:`python{NN}.dll`; it is
    typically installed in ``C:\Windows\System``.  *NN* is the Python version, a
@@ -191,7 +191,7 @@ Embedding the Python interpreter in a Windows app can be summarized as follows:
 2. If you use SWIG, it is easy to create a Python "extension module" that will
    make the app's data and methods available to Python.  SWIG will handle just
    about all the grungy details for you.  The result is C code that you link
-   *into* your .exe file (!)  You do _not_ have to create a DLL file, and this
+   *into* your .exe file (!)  You do **not** have to create a DLL file, and this
    also simplifies linking.
 
 3. SWIG will create an init function (a C function) whose name depends on the
@@ -218,10 +218,10 @@ Embedding the Python interpreter in a Windows app can be summarized as follows:
 5. There are two problems with Python's C API which will become apparent if you
    use a compiler other than MSVC, the compiler used to build pythonNN.dll.
 
-   Problem 1: The so-called "Very High Level" functions that take FILE *
+   Problem 1: The so-called "Very High Level" functions that take ``FILE *``
    arguments will not work in a multi-compiler environment because each
-   compiler's notion of a struct FILE will be different.  From an implementation
-   standpoint these are very _low_ level functions.
+   compiler's notion of a ``struct FILE`` will be different.  From an implementation
+   standpoint these are very low level functions.
 
    Problem 2: SWIG generates the following code when generating wrappers to void
    functions:

Doc/howto/argparse.rst

@@ -761,9 +761,9 @@ your program, just in case they don't know::
    if args.quiet:
        print(answer)
    elif args.verbose:
-       print("{} to the power {} equals {}".format(args.x, args.y, answer))
+       print(f"{args.x} to the power {args.y} equals {answer}")
    else:
-       print("{}^{} == {}".format(args.x, args.y, answer))
+       print(f"{args.x}^{args.y} == {answer}")
 
 Note that slight difference in the usage text. Note the ``[-v | -q]``,
 which tells us that we can either use ``-v`` or ``-q``,