Pinned protobuf version leads to dependency conflict with tensorflow>=2.13 and gRPC on M1/M2

[Zahlii]

tf2onnx>=1.15 pins protobuf~=3.20.2.
Tensorflow >=2.13 requires tf2onnx >= 1.15 due to #2215

In order to use gRPC natively with M1/M2 chips, we need at least grpcio>=1.51.3, which in turn requires protobuf>=4.x

As a result, I can NOT use tensorflow in combination with gRPC on M1/m2, as long as I need tensorflow >= 2.13.

Could we simply unpin the protobuf version in setup.py?

See also #2243

[fatcat-z]

Could you please try to reinstall protobuf to a 4.x version after you install tf2onnx successfully? It might work as well.

IIRC, keeping 3.20.2 is for support tensorflow older version.

[Zahlii]

For my use case, all tests seem to run through (I slightly changed the order, I first install everything including tf2onnx 1.14, then run a update with --no-deps for tf2onnx). However, I am building a library that will be installed by others internally, and I'd rather not have to ask every colleague to manually upgrade tf2onnx after having installed the library.

[Zahlii]

Actually, it seems there is another issue, starting with 2.14 (?) of tensorflow. Flatbuffers changed their version format from 2.xxx to 22.xxx, and tf2onnx seems to be incompatible with newer versions.

186.9 The conflict is caused by:
186.9 tensorflow 2.14.0 depends on flatbuffers>=23.5.26
186.9 tf2onnx 1.14.0 depends on flatbuffers<3.0 and >=1.12

[Zahlii]

197.7 The conflict is caused by:
197.7 tensorflow 2.13.0 depends on flatbuffers>=23.1.21
197.7 tf2onnx 1.14.0 depends on flatbuffers<3.0 and >=1.12

[Ganga-Yadav]

Could you please try to reinstall protobuf to a 4.x version after you install tf2onnx successfully? It might work as well.

IIRC, keeping 3.20.2 is for support tensorflow older version.

i used :
Name: protobuf
Version: 4.25.1 with tensorflow Version: 2.13 but it didn't work ..

[fatcat-z]

197.7 The conflict is caused by: 197.7 tensorflow 2.13.0 depends on flatbuffers>=23.1.21 197.7 tf2onnx 1.14.0 depends on flatbuffers<3.0 and >=1.12

Starts from tf2onnx 1.15.0, flatbuffers is not required to be <3.0.

[fatcat-z]

Could you please try to reinstall protobuf to a 4.x version after you install tf2onnx successfully? It might work as well.
IIRC, keeping 3.20.2 is for support tensorflow older version.

i used : Name: protobuf Version: 4.25.1 with tensorflow Version: 2.13 but it didn't work ..

Could you please share more details about your scenario which doesn't work?

[hmc-cs-mdrissi]

IIRC, keeping 3.20.2 is for support tensorflow older version.

Older tf versions already have constraint blocking protobuf<4. Adding constraint for older version of another package mostly introduces dependency conflicts and pushes towards vendoring the library/re-packaging it entirely. Especially as pip ecosystem lacks a good mechanism for overrides so I'd also find it helpful if constraint is dropped.

[harshad16]

I agree with @hmc-cs-mdrissi insight, the pinned protobuf is causing issue with other python modules, which requires protobuf > 4 . Can the pinned version be dropped form tf2onnx?

[fatcat-z]

I agree with @hmc-cs-mdrissi insight, the pinned protobuf is causing issue with other python modules, which requires protobuf > 4 . Can the pinned version be dropped form tf2onnx?

Thanks for your comments..

At this moment, tf2onnx is still designed to support older tf versions as many as possible and some of them could not work on a version above 3.20.x. In addition, couple of dependencies of tf2onnx also required the protobuf version ~= 3.20.0.

[reyammer]

I hit this problem today: I have a number of dependencies requiring a recent version of protobuf, but tf2onnx requires an old one. But it seems tf2onnx actually would work well with more recent versions. With uv, I solved this problem by adding something like this to pyproject.toml:

[tool.uv]
override-dependencies = ["protobuf==5.29.2"]

Link to uv's docs on the topic.

In this way, the resolver ignores the very restrictive tf2onnx requirement. It's a hack and I wouldn't use it in prod environments, but should be good enough for many other use cases. Hope this helps.