Catch up with master

Author: brandtbucher

Doc/c-api/arg.rst

@@ -149,7 +149,7 @@ which disallows mutable objects such as :class:`bytearray`.
       Previously, :exc:`TypeError` was raised when embedded null code points
       were encountered in the Python string.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.12
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsWideCharString`.
 
@@ -158,23 +158,23 @@ which disallows mutable objects such as :class:`bytearray`.
    Unicode data buffer, the second one its length.  This variant allows
    null code points.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.12
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsWideCharString`.
 
 ``Z`` (:class:`str` or ``None``) [const Py_UNICODE \*]
    Like ``u``, but the Python object may also be ``None``, in which case the
    :c:type:`Py_UNICODE` pointer is set to ``NULL``.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.12
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsWideCharString`.
 
 ``Z#`` (:class:`str` or ``None``) [const Py_UNICODE \*, :c:type:`Py_ssize_t`]
    Like ``u#``, but the Python object may also be ``None``, in which case the
    :c:type:`Py_UNICODE` pointer is set to ``NULL``.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.12
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsWideCharString`.
 

Doc/c-api/unicode.rst

@@ -19,8 +19,7 @@ points must be below 1114112 (which is the full Unicode range).
 
 :c:type:`Py_UNICODE*` and UTF-8 representations are created on demand and cached
 in the Unicode object.  The :c:type:`Py_UNICODE*` representation is deprecated
-and inefficient; it should be avoided in performance- or memory-sensitive
-situations.
+and inefficient.
 
 Due to the transition between the old APIs and the new APIs, Unicode objects
 can internally be in two states depending on how they were created:
@@ -434,7 +433,7 @@ APIs:
 
    If *u* is ``NULL``, this function behaves like :c:func:`PyUnicode_FromUnicode`
    with the buffer set to ``NULL``.  This usage is deprecated in favor of
-   :c:func:`PyUnicode_New`.
+   :c:func:`PyUnicode_New`, and will be removed in Python 3.12.
 
 
 .. c:function:: PyObject *PyUnicode_FromString(const char *u)
@@ -676,7 +675,7 @@ APIs:
 Deprecated Py_UNICODE APIs
 """"""""""""""""""""""""""
 
-.. deprecated-removed:: 3.3 4.0
+.. deprecated-removed:: 3.3 3.12
 
 These API functions are deprecated with the implementation of :pep:`393`.
 Extension modules can continue using them, as they will not be removed in Python

Doc/library/idle.rst

@@ -726,6 +726,15 @@ with objects that get input from and send output to the Shell window.
 The original values stored in ``sys.__stdin__``, ``sys.__stdout__``, and
 ``sys.__stderr__`` are not touched, but may be ``None``.
 
+Sending print output from one process to a text widget in another is
+slower than printing to a system terminal in the same process.
+This has the most effect when printing multiple arguments, as the string
+for each argument, each separator, the newline are sent separately.
+For development, this is usually not a problem, but if one wants to
+print faster in IDLE, format and join together everything one wants
+displayed together and then print a single string.  Both format strings
+and :meth:`str.join` can help combine fields and lines.
+
 IDLE's standard stream replacements are not inherited by subprocesses
 created in the execution process, whether directly by user code or by
 modules such as multiprocessing.  If such subprocess use ``input`` from

Include/methodobject.h

@@ -41,7 +41,13 @@ struct PyMethodDef {
 };
 typedef struct PyMethodDef PyMethodDef;
 
+/* PyCFunction_New is declared as a function for stable ABI (declaration is
+ * needed for e.g. GCC with -fvisibility=hidden), but redefined as a macro
+ * that calls PyCFunction_NewEx. */
+PyAPI_FUNC(PyObject *) PyCFunction_New(PyMethodDef *, PyObject *);
 #define PyCFunction_New(ML, SELF) PyCFunction_NewEx((ML), (SELF), NULL)
+
+/* PyCFunction_NewEx is similar: on 3.9+, this calls PyCMethod_New. */
 PyAPI_FUNC(PyObject *) PyCFunction_NewEx(PyMethodDef *, PyObject *,
                                          PyObject *);
 

Lib/idlelib/NEWS.txt

@@ -1,7 +1,12 @@
-What's New in IDLE 3.10.0 (since 3.9.0)
+What's New in IDLE 3.10.0
+(since 3.9.0)
 Released on 2021-10-04?
-======================================
+=========================
+
 
+bpo-43283: Document why printing to IDLE's Shell is often slower than
+printing to a system terminal and that it can be made faster by
+pre-formatting a single string before printing.
 
 bpo-23544: Disable Debug=>Stack Viewer when user code is running or
 Debugger is active, to prevent hang or crash.  Patch by Zackery Spytz.

Lib/idlelib/help.html

@@ -679,6 +679,14 @@ <h3>Running user code<a class="headerlink" href="#running-user-code" title="Perm
 with objects that get input from and send output to the Shell window.
 The original values stored in <code class="docutils literal notranslate"><span class="pre">sys.__stdin__</span></code>, <code class="docutils literal notranslate"><span class="pre">sys.__stdout__</span></code>, and
 <code class="docutils literal notranslate"><span class="pre">sys.__stderr__</span></code> are not touched, but may be <code class="docutils literal notranslate"><span class="pre">None</span></code>.</p>
+<p>Sending print output from one process to a text widget in another is
+slower than printing to a system terminal in the same process.
+This has the most effect when printing multiple arguments, as the string
+for each argument, each separator, the newline are sent separately.
+For development, this is usually not a problem, but if one wants to
+print faster in IDLE, format and join together everything one wants
+displayed together and then print a single string.  Both format strings
+and <a class="reference internal" href="stdtypes.html#str.join" title="str.join"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.join()</span></code></a> can help combine fields and lines.</p>
 <p>IDLE’s standard stream replacements are not inherited by subprocesses
 created in the execution process, whether directly by user code or by
 modules such as multiprocessing.  If such subprocess use <code class="docutils literal notranslate"><span class="pre">input</span></code> from
@@ -982,7 +990,7 @@ <h3>Navigation</h3>
 <br />
     <br />
 
-    Last updated on Feb 21, 2021.
+    Last updated on Feb 23, 2021.
     <a href="https://docs.python.org/3/bugs.html">Found a bug</a>?
     <br />
 

Lib/test/test_traceback.py

@@ -232,6 +232,29 @@ def test_format_exception_only_exc(self):
         output = traceback.format_exception_only(Exception("projector"))
         self.assertEqual(output, ["Exception: projector\n"])
 
+    def test_exception_is_None(self):
+        NONE_EXC_STRING = 'NoneType: None\n'
+        excfile = StringIO()
+        traceback.print_exception(None, file=excfile)
+        self.assertEqual(excfile.getvalue(), NONE_EXC_STRING)
+
+        excfile = StringIO()
+        traceback.print_exception(None, None, None, file=excfile)
+        self.assertEqual(excfile.getvalue(), NONE_EXC_STRING)
+
+        excfile = StringIO()
+        traceback.print_exc(None, file=excfile)
+        self.assertEqual(excfile.getvalue(), NONE_EXC_STRING)
+
+        self.assertEqual(traceback.format_exc(None), NONE_EXC_STRING)
+        self.assertEqual(traceback.format_exception(None), [NONE_EXC_STRING])
+        self.assertEqual(
+            traceback.format_exception(None, None, None), [NONE_EXC_STRING])
+        self.assertEqual(
+            traceback.format_exception_only(None), [NONE_EXC_STRING])
+        self.assertEqual(
+            traceback.format_exception_only(None, None), [NONE_EXC_STRING])
+
 
 class TracebackFormatTests(unittest.TestCase):
 

Lib/test/test_xml_etree.py

@@ -108,6 +108,19 @@
 <document>&entity;</document>
 """
 
+ATTLIST_XML = """\
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE Foo [
+<!ELEMENT foo (bar*)>
+<!ELEMENT bar (#PCDATA)*>
+<!ATTLIST bar xml:lang CDATA "eng">
+<!ENTITY qux "quux">
+]>
+<foo>
+<bar>&qux;</bar>
+</foo>
+"""
+
 def checkwarnings(*filters, quiet=False):
     def decorator(test):
         def newtest(*args, **kwargs):
@@ -1354,6 +1367,12 @@ def test_tree_write_attribute_order(self):
         self.assertEqual(serialize(root, method='html'),
                 '<cirriculum status="public" company="example"></cirriculum>')
 
+    def test_attlist_default(self):
+        # Test default attribute values; See BPO 42151.
+        root = ET.fromstring(ATTLIST_XML)
+        self.assertEqual(root[0].attrib,
+                         {'{http://www.w3.org/XML/1998/namespace}lang': 'eng'})
+
 
 class XMLPullParserTest(unittest.TestCase):
 

Lib/traceback.py

@@ -91,7 +91,10 @@ def _parse_value_tb(exc, value, tb):
     if (value is _sentinel) != (tb is _sentinel):
         raise ValueError("Both or neither of value and tb must be given")
     if value is tb is _sentinel:
-        return exc, exc.__traceback__
+        if exc is not None:
+            return exc, exc.__traceback__
+        else:
+            return None, None
     return value, tb
 
 
@@ -528,7 +531,9 @@ def __init__(self, exc_type, exc_value, exc_traceback, *, limit=None,
                     cause = None
 
                 if compact:
-                    need_context = cause is None and not e.__suppress_context__
+                    need_context = (cause is None and
+                                    e is not None and
+                                    not e.__suppress_context__)
                 else:
                     need_context = True
                 if (e and e.__context__ is not None

Lib/xml/etree/ElementTree.py

@@ -1560,7 +1560,6 @@ def __init__(self, *, target=None, encoding=None):
         # Configure pyexpat: buffering, new-style attribute handling.
         parser.buffer_text = 1
         parser.ordered_attributes = 1
-        parser.specified_attributes = 1
         self._doctype = None
         self.entity = {}
         try:
@@ -1580,7 +1579,6 @@ def _setevents(self, events_queue, events_to_report):
         for event_name in events_to_report:
             if event_name == "start":
                 parser.ordered_attributes = 1
-                parser.specified_attributes = 1
                 def handler(tag, attrib_in, event=event_name, append=append,
                             start=self._start):
                     append((event, start(tag, attrib_in)))

Misc/NEWS.d/next/C API/2021-02-16-17-30-16.bpo-43239.FQqOGz.rst

@@ -0,0 +1,2 @@
+The :c:func:`PyCFunction_New` function is now exported in the ABI when
+compiled with ``-fvisibility=hidden``.

Misc/NEWS.d/next/Documentation/2020-06-15-10-45-45.bpo-36346.H0sS_i.rst

@@ -0,0 +1,2 @@
+Update some deprecated unicode APIs which are documented as "will be removed
+in 4.0" to "3.12". See :pep:`623` for detail.

Misc/NEWS.d/next/IDLE/2021-02-21-16-30-10.bpo-43283.DLBwYn.rst

@@ -0,0 +1,3 @@
+Document why printing to IDLE's Shell is often slower than printing to a
+system terminal and that it can be made faster by pre-formatting a single
+string before printing.

Misc/NEWS.d/next/Library/2020-10-26-18-01-09.bpo-42151.et5f7s.rst

@@ -0,0 +1,3 @@
+Make the pure Python implementation of :mod:`xml.etree.ElementTree` behave
+the same as the C implementation (:mod:`_elementree`) regarding default
+attribute values (by not setting ``specified_attributes=1``).

Misc/NEWS.d/next/Library/2021-02-06-21-21-35.bpo-43146.MHtb2v.rst

@@ -0,0 +1 @@
+Fix recent regression in None argument handling in :mod:`~traceback` module functions.

Misc/NEWS.d/next/Library/2021-02-23-17-20-16.bpo-43146.JAFplg.rst

@@ -0,0 +1 @@
+Handle None in single-arg versions of :func:`~traceback.print_exception` and :func:`~traceback.format_exception`.