-
Notifications
You must be signed in to change notification settings - Fork 699
Wrong scheduling because of the lack of memory dependencies #1283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Problem illustrated in #1284. This PR adds two test cases that build exactly the same graph. However, by just changing the order of the creation of the nodes, we can trick the scheduler to produce an invalid schedule. |
After talking with @nadavrot this morning, we are going to explicit state that save nodes happen after the last use of the related variable. In other words, referencing a variable will always get you the old value. Similarly we are going to enforce that every variable has at most one save nodes. That way it will remove the ambiguity of which save should happen last (all the others would have been dead because of the other restriction.) |
As expected while reviewing #1176, I realize that our implicit memory dependencies are going to bite us again during graph partitioning. The problem there is that the post order visiting methods don't see the memory dependencies edges. So, we will have to do one of the following fix the visiting scheme, come up with targeted fix for the graph partitioning stuff, or model the memory dependencies. |
When a variable is used as output in a save node, this is expected to be its last use. Make sure the GraphSchedule looks at the implicit dependencies between all the uses of a variable and its save node when walking the dependencies. This fixes issue #1283.
Fixed by #1307. |
I'm going to send a PR to show how we can fool the scheduler.
The text was updated successfully, but these errors were encountered: