SSL issue starting from openssl 3.2

[nurfed1]

Bug report

Bug description:

Hi,

There appears to be an issue either in the asyncio SSL code when upgrading a connection with start_tls or in the function _ssl__SSLSocket_read_impl, starting from OpenSSL 3.2.

The following code works fine with OpenSSL 3.1.4 but fails with OpenSSL 3.2.0 when TLS 1.3 is used. I’ve tested multiple combinations of OpenSSL and Python versions, and the issue seems to lie either in OpenSSL or Python. I’m unsure whether to report this here or on the OpenSSL GitHub, but it does appear to be a bug.

Starting with OpenSSL 3.2, the following exception is raised:
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2576)

This issue is caused by SSL_read_ex returning SSL_ERROR_SYSCALL after successfully reading the HTTP response body. From my observations, errno is 0 at the time of the error.

The OpenSSL 1.1.1 documentation (SSL_get_error) suggests this behavior should not occur in modern versions of OpenSSL. If I understand correctly, this could also be a regression introduced in OpenSSL 3.2.

The SSL_ERROR_SYSCALL with errno value of 0 indicates unexpected EOF from the peer. This will be properly 
reported as SSL_ERROR_SSL with reason code SSL_R_UNEXPECTED_EOF_WHILE_READING in the OpenSSL 3.0 
release because it is truly a TLS protocol error to terminate the connection without a SSL_shutdown().

The issue is kept unfixed in OpenSSL 1.1.1 releases because many applications which choose to ignore this 
protocol error depend on the existing way of reporting the error.

I’m not sure if this is a bug in Python or OpenSSL, but it seems likely to become a problem once more systems upgrade to OpenSSL 3.2.

Here's the minimal reproducible sample code:

import asyncio
import ssl


class HttpProxyClient:
    def __init__(self, proxy_host, proxy_port, target_host, target_port):
        self.proxy_host = proxy_host
        self.proxy_port = proxy_port
        self.target_host = target_host
        self.target_port = target_port

    async def connect(self):
        reader, writer = await asyncio.open_connection(self.proxy_host, self.proxy_port)

        connect_request = (
            f"CONNECT {self.target_host}:{self.target_port} HTTP/1.1\r\n"
            f"Host: {self.target_host}:{self.target_port}\r\n"
            f"Proxy-Connection: keep-alive\r\n\r\n"
        )
        writer.write(connect_request.encode())
        await writer.drain()

        response = await reader.read(4096)

        if b'200 Connection established' not in response:
            writer.close()
            await writer.wait_closed()
            raise Exception("Failed to establish connection with the proxy.")

        ctx = ssl.create_default_context(
            purpose=ssl.Purpose.SERVER_AUTH
        )
        # Disable checks for docker test container
        ctx.check_hostname = False
        ctx.verify_mode = ssl.CERT_NONE

        # Disable tls1.3: this fixes the issue with openssl 3.2+
        # ctx.options |= ssl.OP_NO_TLSv1_3

        # Disable everything except tls1.3: this causes a crash with openssl 3.2+
        ctx.options |= ssl.OP_NO_TLSv1_2 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3

        await writer.start_tls(ctx)

        print("Upgraded to SSL/TLS")

        return reader, writer

    async def run(self):
        http_request = (
            f"GET / HTTP/1.1\r\n"
            f"Host: {self.target_host}\r\n"
            f"Accept: */*\r\n"
            f"Connection: close\r\n\r\n"
        )
        try:
            reader, writer = await self.connect()

            writer.write(http_request.encode())
            await writer.drain()

            while not reader.at_eof():
                response = await reader.read(4096)
                print(response.decode())

            writer.close()
            await writer.wait_closed()
        except Exception as e:
            raise


async def main():
    proxy_host = "localhost"
    proxy_port = 9001
    target_host = "google.com"
    target_port = 443

    client = HttpProxyClient(proxy_host, proxy_port, target_host, target_port)
    await client.run()

asyncio.run(main())

CPython versions tested on:

3.10, 3.11, 3.12, 3.13, 3.14

Operating systems tested on:

Linux

[keepworking]

Hello nurfed1,

I tried to run provided test code and it works on python v3.13.1

orange@32thread-server:~/project/python-contribute/cpython/build$ ./python test.py
/home/orange/project/python-contribute/cpython/build/test.py:41: DeprecationWarning: ssl.OP_NO_SSL*/ssl.OP_NO_TLS* options are deprecated
  ctx.options |= ssl.OP_NO_TLSv1_2 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
Upgraded to SSL/TLS
HTTP/1.1 301 Moved Permanently
Location: https://www.google.com/
Content-Type: text/html; charset=UTF-8
Content-Security-Policy-Report-Only: object-src 'none';base-uri 'self';script-src 'nonce-fTBc8ST-VheCPAr8YOXdYw' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
Date: Tue, 14 Jan 2025 09:59:42 GMT
Expires: Thu, 13 Feb 2025 09:59:42 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 220
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Connection: close

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>

And I Cannot find same Issue on another version of python (v3.10, v3.9)

orange@32thread-server:~/project/python-contribute/cpython/build$ python3.9 test.py
Traceback (most recent call last):
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 81, in <module>
    asyncio.run(main())
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 79, in main
    await client.run()
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 57, in run
    reader, writer = await self.connect()
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 43, in connect
    await writer.start_tls(ctx)
AttributeError: 'StreamWriter' object has no attribute 'start_tls'
1 orange@32thread-server:~/project/python-contribute/cpython/build$ python3.10 test.py
/home/orange/project/python-contribute/cpython/build/test.py:41: DeprecationWarning: ssl.OP_NO_SSL*/ssl.OP_NO_TLS* options are deprecated
  ctx.options |= ssl.OP_NO_TLSv1_2 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
Traceback (most recent call last):
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 81, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 79, in main
    await client.run()
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 57, in run
    reader, writer = await self.connect()
  File "/home/orange/project/python-contribute/cpython/build/test.py", line 43, in connect
    await writer.start_tls(ctx)
AttributeError: 'StreamWriter' object has no attribute 'start_tls'

Here is my openssl version.

orange@32thread-server:~/project/python-contribute/cpython/build$ openssl version -a
OpenSSL 3.2.4-dev  (Library: OpenSSL 3.2.4-dev )
built on: Tue Jan 14 10:23:08 2025 UTC
platform: linux-x86_64
options:  bn(64,64)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG
OPENSSLDIR: "/usr/local/ssl"
ENGINESDIR: "/usr/local/lib/engines-3"
MODULESDIR: "/usr/local/lib/ossl-modules"
Seeding source: os-specific
CPUINFO: OPENSSL_ia32cap=0x7ed8320b078bffff:0x40069c219c97a9

In my Opinion, It may already fiexed in this follows issue. #115627
This changes applied in python 3.13.1.

[nurfed1]

Hi keepworking,

While this issue seems potentially related, I don’t believe it resolves the problem.

I just ran a simple test case:

docker run --rm -it -v $PWD:/app archlinux:latest bash
pacman -Syu python
[root@a179afe7470f /]# python -V
Python 3.13.1
[root@a179afe7470f /]# openssl version
OpenSSL 3.4.0 22 Oct 2024 (Library: OpenSSL 3.4.0 22 Oct 2024)

[root@a179afe7470f /]# python /app/test.py
/app/test.py:41: DeprecationWarning: ssl.OP_NO_SSL*/ssl.OP_NO_TLS* options are deprecated
  ctx.options |= ssl.OP_NO_TLSv1_2 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
Upgraded to SSL/TLS
HTTP/1.1
Traceback (most recent call last):
  File "/app/test.py", line 81, in <module>
    asyncio.run(main())
    ~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/asyncio/base_events.py", line 720, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/app/test.py", line 79, in main
    await client.run()
  File "/app/test.py", line 63, in run
    response = await reader.read(4096)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/asyncio/streams.py", line 730, in read
    await self._wait_for_data('read')
  File "/usr/lib/python3.13/asyncio/streams.py", line 539, in _wait_for_data
    await self._waiter
  File "/usr/lib/python3.13/asyncio/sslproto.py", line 740, in _do_read
    self._do_read__copied()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/asyncio/sslproto.py", line 785, in _do_read__copied
    chunk = self._sslobj.read(self.max_size)
  File "/usr/lib/python3.13/ssl.py", line 868, in read
    v = self._sslobj.read(len)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2638)

[root@a179afe7470f /]# cat /app/test.py 
import asyncio
import ssl


class HttpProxyClient:
    def __init__(self, proxy_host, proxy_port, target_host, target_port):
        self.proxy_host = proxy_host
        self.proxy_port = proxy_port
        self.target_host = target_host
        self.target_port = target_port

    async def connect(self):
        reader, writer = await asyncio.open_connection(self.proxy_host, self.proxy_port)

        connect_request = (
            f"CONNECT {self.target_host}:{self.target_port} HTTP/1.1\r\n"
            f"Host: {self.target_host}:{self.target_port}\r\n"
            f"Proxy-Connection: keep-alive\r\n\r\n"
        )
        writer.write(connect_request.encode())
        await writer.drain()

        response = await reader.read(4096)

        if b'200 Connection established' not in response:
            writer.close()
            await writer.wait_closed()
            raise Exception("Failed to establish connection with the proxy.")

        ctx = ssl.create_default_context(
            purpose=ssl.Purpose.SERVER_AUTH
        )
        # Disable checks for docker test container
        ctx.check_hostname = False
        ctx.verify_mode = ssl.CERT_NONE

        # Disable tls1.3: this fixes the issue with openssl 3.2+
        # ctx.options |= ssl.OP_NO_TLSv1_3

        # Disable everything except tls1.3: this causes a crash with openssl 3.2+
        ctx.options |= ssl.OP_NO_TLSv1_2 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3

        await writer.start_tls(ctx)

        print("Upgraded to SSL/TLS")

        return reader, writer

    async def run(self):
        http_request = (
            f"GET / HTTP/1.1\r\n"
            f"Host: {self.target_host}\r\n"
            f"Accept: */*\r\n"
            f"Connection: close\r\n\r\n"
        )
        try:
            reader, writer = await self.connect()

            writer.write(http_request.encode())
            await writer.drain()

            while not reader.at_eof():
                response = await reader.read(4096)
                print(response.decode())

            writer.close()
            await writer.wait_closed()
        except Exception as e:
            raise


async def main():
    proxy_host = "172.17.0.1"
    proxy_port = 9001
    target_host = "google.com"
    target_port = 443

    client = HttpProxyClient(proxy_host, proxy_port, target_host, target_port)
    await client.run()

asyncio.run(main())

In the test, I used Burp as an HTTP proxy on 172.17.0.1:9001.

From my review of the Python 3.13.1 source used by Arch Linux, this version already contains the gh-115627 fix mentioned in #115627. Therefore, I don't believe the issue is fully resolved.

[keepworking]

Thanks to your test.

I want to request more test to you.

The Provided log shows an ssl version to v3.4.0, can you test by ssl v3.2 ?

I will test it with burp later.

[nurfed1]

Yes, Arch Linux is currently using OpenSSL 3.4, but I recall conducting tests with various Python versions (3.10, 3.11, 3.12, 3.13, and 3.14) in combination with different OpenSSL versions (3.0.9, 3.1.4, 3.2.0, and 3.4.0).

I noticed the issues started with OpenSSL 3.2.0, specifically with TLS 1.3, which seems to have become the default as of that version.

Testing was a bit of a hassle, having to compile various OpenSSL and Python versions.

[encukou]

The reproducer requires a proxy running on port 9001, right? What would be the best way to get that running in a container?