[GraphOptimizer] Fix bug in post-lowering optimizations for OpenCL #3493
Assignees
Comments
|
Great catch @shajrawi ! |
|
Found out why DCE deletes the Reshape node, at some point the conv. is no longer its user: it is done in the |
shajrawi
added a commit
to shajrawi/glow
that referenced
this issue
Nov 6, 2019
Fixes pytorch#3452 Also Fixes pytorch#3493 and pytorch#3500 GraphOptimizer bugs which were found after adding the layout verifier. Provides a workaround for the pytorch#3499 issue which was also found via the verifier. Note: I did not want to break the `enum ConvolutionLayout` introduced in 5074a72, As such, I used it in the verifier / did not change the creation of said nodes. HOWEVER: We should use the more-generic string-based layout, which I introduce to Transpose node in this commit: it is basically an extendable enum that can be used in the backends without touching the generic code base. as a bonus, it makes differentiation easier: see how it is done for transpose now in `Function *glow::differentiate`. Getting rid of said enum is a proposed TODO / follow-up. Also note that some nodes *need* layout requirements, which have been added, namely we need to know the layout for placeholders and constants (obviously) and for reshapes (in case we optimized a transpose into a reshape. An additional nice-to-have feature of the string-based layout is the wildcard / any-layout option. Some operations, such as data parallel nodes, might accept any layout. A potential follow-up is to get create a "Solver" that automatically inserts transposes if the layouts do not match, this might greatly simplify the loader: we no longer need to insert transposes based on if we are importing NHWC or NCHW (for example). We just need to annotate the placeholder with the layout information we've get at load-time, and which we "forget" afterwards. The verifier is useful even without creating said solver, it exposed a couple of bugs which are mentioned in this commit, as such any proposed solvers are not a must-have to demonstrate the usefulness of this commit.
vdantu
pushed a commit
to vdantu/glow
that referenced
this issue
Jul 12, 2020
…pytorch#3503) Summary: Note: I did not want to break the `enum ConvolutionLayout` introduced in 5074a72, As such, I used it in the verifier / did not change the creation of said nodes. HOWEVER: We should use the more-generic string-based layout, which I introduce to Transpose node in this commit: it is basically an extendable enum that can be used in the backends without touching the generic code base. as a bonus, it makes differentiation easier: see how it is done for transpose now in `Function *glow::differentiate`. Getting rid of said enum is a proposed TODO / follow-up. Also note that some nodes *need* layout requirements, which have been added, namely we need to know the layout for placeholders and constants (obviously) and for reshapes (in case we optimized a transpose into a reshape). An additional nice-to-have feature of the string-based layout is the wildcard / any-layout option. Some operations, such as data parallel nodes, might accept any layout. A potential follow-up is to get create a "Solver" that automatically inserts transposes if the layouts do not match, this might greatly simplify the loader: we no longer need to insert transposes based on if we are importing NHWC or NCHW (for example). We just need to annotate the placeholder with the layout information we've get at load-time, and which we "forget" afterwards. The verifier is useful even without creating said solver, it exposed a couple of bugs which are mentioned in this commit, as such any proposed solvers are not a must-have to demonstrate the usefulness of this commit. Fixes pytorch#3452 Also Fixes pytorch#3493 and Fixes pytorch#3500 GraphOptimizer bugs which were found after adding the layout verifier. Provides a workaround for the pytorch#3499 issue which was also found via the verifier. Pull Request resolved: pytorch#3503 Test Plan: `ninja test` Differential Revision: D18357369 Pulled By: shajrawi fbshipit-source-id: 45f91fbe120b234c2a85879cee9ee0de6c100b50
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As part of #3452 I created a layout verifier for the
OpenCLbackend. It caught a bug in our optimizations: seesmallConvbackend correctness test with theOpenCLbackend for a tiny repro. We have aNHWCconv. which we turn intoNCHWwith a transpose of the inputs (which is fine). then the bug occurs when we optimize post lowering:OptimizeTransposeIntoReshapeturns the transpose (of a placeholder) into a reshape andDCEremoves the reshape. this is obviously incorrect because the placeholder is inNHWCformat.The text was updated successfully, but these errors were encountered: