Eventlet retirement

[nzlosh]

SUMMARY

As reported by @cognifloyd in slack: the eventlet team has announced their intention to retire the project. https://github.com/eventlet/eventlet?tab=readme-ov-file#warning This will have significant implications for the StackStorm project as the code base is architected and designed around eventlet and the synchronous programming model.

[guzzijones]

gevent has monkey patching.

[cognifloyd]

I prefer using standardized asyncio.

I consider monkey patching to be an anti-pattern as it makes debugging very difficult (was it actually monkey patched? Did something reload the code effectively undoing the monkey patch?).

Another reason to use asyncio: Once we implement StackStorm/community#8 the chatops layer will use asyncio since OpsDroid uses asyncio.

I think the way to migrate to asyncio is to migrate one service at a time. That might mean we have a separate async-compatible st2common-compatible library, slowly migrating the dependencies of each service from the sync to the async lib.

Also, the eventlet folks are working on tools to allow asyncio and eventlet to live in the same process, thus smoothing out the migration path. Depending on how that goes, and when it completes, that might also be a big help:

• Asyncio integration for eventlet eventlet/eventlet#868

As far as gevent: That will probably be a shorter leap, as support is already partially stubbed in. Any direct imports of eventlet should migrate to using this file instead (if possible):
https://github.com/StackStorm/st2/blob/master/st2common/st2common/util/concurrency.py
And the monkey_patch code would need to be updated to support gevent:
https://github.com/StackStorm/st2/blob/master/st2common/st2common/util/monkey_patch.py

[guzzijones]

I also dislike monkey patching and would prefer a service by service migration to asyncio.

[Robxxt]

Hi there, I would like to help to do the migration. How should we start?

[nzlosh]

@ZoeLeah @guzzijones You've both expressed interest in working on this issue. I recommend organising a meeting between the people who have expressed interest to work on this task and formulate an execution plan. It'd be good to document the exit strategy, the required steps and the overall changes required to move off eventlets.

The question is still open about the optimal exit strategy, given the lack of resources the StackStorm project is currently suffering from, would it be a more optimal path to switch to gevent (to switch to an actively maintained asynchronous framework) and then migrate service by service to aiohub (eventlet/aiohub#2) in the long term? Or would it better to switch straight to asyncio using the eventlet aiohub directly?

[Robxxt]

@nzlosh Through which platform do you suggest to have the meeting?
Do you have any date in mind?
It would fit me something from Monday to Friday in this time space: 8am-4pm UTC+2(Berlin time)

[AndroxxTraxxon]

Hi, I haven't seen any activity in this space, Is there a branch or fork where this work is being completed?

[Robxxt]

Hi @AndroxxTraxxon as far as I know there is not any branch. We haven't even decided in the optimal strategy to make the migration to asyncio😅

[AndroxxTraxxon]

I'm liking the strategy proposed by @guzzijones of migrating service by service over to asyncio. would we want to also jump over to an asgi service handler like hypercorn as well? It appears that gunicorn also depends on eventlet, and based on past challenges that package has faced (like being abandoned for 2+ years), it may be best to remove that dependency for one that is more actively maintained

[Robxxt]

How should we proceed with the service by service migration. Should we use eventlet_asyncio hub? Or directly re-write eventlet functionallity in it's counterpart in asyncio??