Skip to content

Conversation

DiTo97
Copy link
Collaborator

@DiTo97 DiTo97 commented May 10, 2024

#199

  • wrap sequential functions into async equivalents
  • seamless event loop handling
  • batch size for semaphore not to exhaust too many resources

TODO - still untested

…phore for parallel graph instances

TODO: still untested
@Kingki19
Copy link

I hope it's fix error: RuntimeError: asyncio.run() cannot be called from a running event loop in smart_scraper_graph.run()

@PeriniM
Copy link
Contributor

PeriniM commented May 13, 2024

TODO 🐱

  • test it with different batchsizes and system configuration;
  • update docs with benchmarks.

@PeriniM PeriniM merged commit 62a74a5 into ScrapeGraphAI:pre/beta May 13, 2024
Copy link

🎉 This PR is included in version 0.11.0-beta.8 🎉

The release is available on:

Your semantic-release bot 📦🚀

@PeriniM PeriniM linked an issue May 13, 2024 that may be closed by this pull request
Copy link

🎉 This PR is included in version 0.11.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prallel Execution of Nodes
3 participants