Drop mbedTLS and migrate to OpenSSL

[fxcoudert]

There has already been an issue that proposed migration to BoringSSL (#45856), which is not what I propose here.

• Julia already depends on OpenSSL
• Julia currently depends on mbedTLS for:
  • libssh2, on all targets
  • libgit2, on Linux/FreeBSD
  • curl, on Linux/FreeBSD
• All three dependencies could migrate to OpenSSL
• We would cut down the dependency tree
• Avoid relying on an old branch of mbedTLS

From a security perspective, this seems great (fewer updates to manage, and track record shows mbedTLS is frequently behind in Julia).

Are there any downsides to migrating to OpenSSL and removing mbedTLS?

If not, and the idea has support, I volunteer to handle the migration PRs in Yggdrasil and julia.

[giordano]

Julia already depends on OpenSSL

Uhm, I don't think Julia itself does. Some packages in the ecosystem do.

Are there any downsides to migrating to OpenSSL and removing mbedTLS?

Isn't OpenSSL plagued by security vulnerabilities? Last October all stable versions were withdrawn because of high security vulnerabilities, and the fix came only three weeks afterwards: https://www.openssl.org/news/newslog.html.

Pinging some people who may have opinions: @mkitti @eschnett @quinnj

[DilumAluthge]

Also cc: @StefanKarpinski @staticfloat

[mkitti]

I have significantly soured on mbedTLS in that they have no commitment to maintain the binary interface between minor releases.

With OpenSSL moving to semantic versioning with the version 3 release. I think we should seriously consider using OpenSSL v3 instead of mbedTLS. We need to finish moving the ecosystem to OpenSSL v3 first though.

In a similar vein, we should also consider libssh instead of libssh2.

[mkitti]

xref: #43677 (comment)

Here is an excerpt from the mbedtls release notes:

Some fields of mbedtls_ssl_session and mbedtls_ssl_config are in a different order. This only affects applications that define such structures directly or serialize them.

https://github.com/Mbed-TLS/mbedtls/releases/tag/v2.28.0

[fxcoudert]

My 20 cents: OpenSSL has had hiccups in the past, but it seems to me much more stable and backward compatible that mbedTLS. It's also used more widely: at Homebrew, we have more than 10% of our software that depends directly on OpenSSL (and the majority of the packages have some indirect dependence on it).

[mkitti]

mbedTLS long term support

https://github.com/Mbed-TLS/mbedtls/blob/development/BRANCHES.md

We use Semantic Versioning. In particular, we maintain API compatibility in the master branch across minor version changes (e.g. the API of 3.(x+1) is backward compatible with 3.x). We only break API compatibility on major version changes (e.g. from 3.x to 4.0). We also maintain ABI compatibility within LTS branches; see the next section for details.

For the LTS branches, additionally we try very hard to also maintain ABI compatibility (same definition as API except with re-linking instead of re-compiling) and to avoid any increase in code size or RAM usage, or in the minimum version of tools needed to build the code. The only exception, as before, is in case those goals would conflict with fixing a security issue, we will put security first but provide a compatibility option. (So far we never had to break ABI compatibility in an LTS branch, but we occasionally had to increase code size for a security fix.)

mbedTLS Summary

• ABI compatability is only guaranteed in LTS releases
• API compatability is guaranteed with major version releases (semantic versioning is only being applied to the API)
• mbedlts version 2.28 is current LTS, and is supported for 3 years. 2.28 was released in December 2021 release and will be supported until December 2024.

OpenSSL long term support

https://www.openssl.org/policies/releasestrat.html

As of release 3.0.0, the OpenSSL versioning scheme is changing to a more contemporary format: MAJOR.MINOR.PATCH With this format, API/ABI compatibility will be guaranteed for the same MAJOR version number.
This more closely aligns with the expectations of users who are familiar with semantic versioning. However, we have not adopted semantic versioning in the strict sense of its rules, because it would mean changing our current LTS policies and practices.
Version 3.0 will be supported until 2026-09-07 (LTS).
We may designate a release as a Long Term Support (LTS) release. LTS releases will be supported for at least five years and we will specify one at least every four years. Non-LTS releases will be supported for at least two years.

No API or ABI breaking changes are allowed in a minor or patch release. The following stability rules apply to all changes made to code targeted for a major release from version 3.0.0 or later:

• No existing public interface can be modified except where changes are unlikely to break source compatibility or where structures are made opaque.
• No existing public interface can be removed until its replacement has been in place in an LTS stable release. The original interface must also have been documented as deprecated for at least 5 years. A public interface is any function, structure or macro declared in a public header file.
• When structures are made opaque, any newly required accessor macros or functions are added in a feature release of the extant LTS release and all supported intermediate successor releases.

OpenSSL Summary

• ABI and API compatability is guaranteed with the same major version number.
• Stable and well defined public interface in LTS with deprecation warning policy
• OpenSSL version 3.0 is LTS will be supported until 2026-09-07 (LTS).

[ViralBShah]

A major problem we had was not sticking with their LTS versions. I believe 2.28 is an LTS version.

[mkitti]

Yes, but the problem is that "long term" for mbedtls is less than 2 years from now. Will we release another Julia LTS by then?

[ViralBShah]

I imagine we most likely will. Just a personal view.

[joa-quim]

GMT doesn't run on some linuxes because external binaries may depend on OpenSSL
#48419

[mkitti]

I'm not sure if this will solve GMT's issues regarding conflicts with system libraries. You might have to contend with multiple versions of OpenSSL libraries then.

The only ones who can really solve that issue are the system package managers. The alternative is to create our own "system" (e.g. the BinaryBuilder JLLs, containers, or conda-forge).

[joa-quim]

That problem is occurring with binaries built with Conda.

[mkitti]

Right. So conda (probably conda-forge) would need to figure out how to build julia with libraries configured to fit in the conda-forge ecosystem.

[mkitti]

In conda-forge, I think they might build with curl, libgit2, and libssh2 that uses OpenSSL v3, so maybe that works with GMT?

I happen to be one of the contributors to the julia-feedstock.

https://github.com/conda-forge/julia-feedstock/blob/main/recipe/meta.yaml

https://github.com/conda-forge/julia-feedstock/blob/main/recipe/build.sh#L49