[discussion] Add engine version feature flags

[ghost]

This idea was proposed in Discord a few days ago, but it wasn't thoroughly discussed at the moment. This is basically intended to be an escape hatch in the semver system, that would allow us to (eventually) reach stability while keeping room for ourselves to add support for future major versions of the engine, which might require arbitrary changes in the library. This would work in the following way:

• At 0.9 release, add a godot_3_2 feature flag, without making it default. Using impl-proc-macros, we emit a compile error if it is not enabled. This means that current users will be forced to enable the godot_3_2 feature in their Cargo.tomls manually.
• When we add support for Godot 4.0 later, we add a new unstable_godot_4_0 feature flag, which is not covered by the semver guarantees. Again, we use impl-proc-macros to assert that there is one and only one major version enabled. Compatibility will not be broken for 3.2 users since they already enable godot_3_2 manually.
• When we're satisfied with the 4.0 interface, we can now rename unstable_godot_4_0 to godot_4_0, adding stability guarantees to the API.
• Repeat for all future stable engine branches.

This would mean that, even in the case that a future version would require major structural changes to the library, we would still be able to handle it without forking or breaking stability for other versions: we can simply separate core into two internal crates and re-export depending on the feature flag.

This would also make #291 non-breaking because gdnative-bindings can also re-export depending on feature flags, removing the need to deprecate it.

Thoughts are welcome!

[karroffel]

I don't think it makes much sense to decouple version of godot-rust from the version of Godot. In theory all GDNative code should be working with any higher minor and patch version.

Class bindings are not really stable at all and can be regenerated pretty easily (minus breaking changes in the JSON structure maybe), so I maybe we can just sync those up with the godot-rust version + the Godot version (like 0.9.0-godot_3_2_2?)

The core wrappers and NativeScript extension interface are very much tied to major or even minor versions of Godot and I think trying to find stability independently from that will be very tricky.

One crate I can think of that behaves similarly is the gitlab crate. Since they represent the GitLab API and need to wrap it, changes in the way the API works will need changes in the crate.

Dealing with specific versions of a crate for specific versions of the thing that's wrapped is more flexible and will not need as many conditional compilation flags etc.

So I personally think we should keep separate versions of godot-rust for different Godot versions around in branches and backport changes if needed rather than trying to unify the set of APIs that might remain stable.

[ghost]

The issue with branches is that they copy code unconditionally, and thus take a lot of effort to maintain. With conditional compilation, however, code is only duplicated when necessary, while the ability to duplicate code wholesale is preserved when it comes to that. If a future engine version would require large, divergent changes to the core bindings, we may simply separate gdnative-core into two internal crates, and re-export them depending on feature flags. In my opinion, this is strictly more flexible compared to branches.

Furthermore, keeping a single branch allows us to easily distribute the burden of maintaining multiple versions to all contributors. When making changes, contributors can easily identify places where changes need to be adapted to multiple engine versions, and with CI, we can easily guard against regressions, leaving it up to the contributors to fix them. Contrast that with branch backporting, which requires major maintainer effort in identifying and backporting changes, is a hard thing to ask from contributors (seen as extra work), and can often result in merge conflicts that no one wants to deal with.

So why would we even want to support past versions of the engine in the first place, despite limited maintainer resources?

The problem lies in the lifetime of game code. Games (well, non-jam ones) are developed over long time spans, often counted in years, both before and after release. As a game developer, jumping from one major version of the engine to another isn't something I'd usually consider doing in the middle of an ongoing project. With the breadth and depth of breaking changes that will be present in Godot 4.0, upgrading can easily take weeks, or even months of development time even for moderately complex projects. Even after release, games may still need to be supported for significant time spans, which further complicates the problem.

Personally, while I'd love to start a new game project months and maybe years later, and see godot-rust supporting the latest and greatest, the more important thing to me at this moment is still solid, maintained 3.2 support. The projects in this community may be small or "abstract" for now, but eventually some of them will grow into large, "concrete" ones, and the same problem will inevitably rear its ugly head.

Anyway, I hope that explains my motivations for making this proposal as a maintainer, as a contributor, and as a user. I have moved it to the 0.10 milestone as I don't believe it's immediately necessary for now, but this needs to be eventually addressed before we try to add 4.0 support. Meanwhile, I'm open to alternative schemes that 1) don't involve divergent branching or otherwise excessive maintainer work, and 2) allow us to keep 3.2 support maintained. If you have any ideas, please feel free to suggest it!

[Bromeon]

Closing for now. Whether this is feasible, depends on the new GDExtensions API and how much it differs from GDNative.
We should re-evaluate it once we have more information about GDExtensions.