Soften name validation warnings to avoid CI issues #717
Conversation
In fe34619 we degraded the hard error (ie. a `throw`) for non-compliant field names (ie. starting with `__`) to a `console.error`. We've found, however, that certain CI systems will treat the use of `console.error` as a cause for failure, and our intent here is not to block what would otherwise be a valid test run. Switch to `console.warn` instead. Eventually in a later release this will become a hard error, but for now continuing to treat this like a "deprecation" and just warn is appropriate.
This goes a step further than the parent commit, which downgrades `console.error` to `console.warn` for cases that are really just advisory "deprecation" warnings for non-compliant schemas. The motivation in that commit was preventing CI failures stemming from use of `console.error`. That may not be enough though, because on some JS engines (notably Chrome/Node), "Error: " still appears in the output thanks to our use of `Error` objects to get stack traces. Depending on how the CI is set-up, that could also be enough to spuriously fail the run. So in this commit, we add a `formatWarning` helper that takes the `Error` object and grooms it for use as a human-readable warning message. Added tests to capture and verify the different engine behaviors.
To adequately serve as a deprecation warning, the message should provide advance notice that it non-compliance will eventually become a hard error. I haven't put a version number in there yet because I don't know which version it will happen in. Once we make that call we should make the message more precise.
|
Thanks for doing the research and adding tests. This looks great. Feel free to merge. I think beyond this we should follow-up with with a slightly more invasive change to separate the construction of a schema from the validation of a schema. When building a "source of truth" schema (e.g. you run a node.js server, and use GraphQL.js to present a GraphQL service) then we should encourage validation - strict validation - to ensure the schema is well formed. I'm not sure what the best way to accomplish this would be and am open to ideas. Perhaps When building a "reference" schema (e.g. you used introspection to learn about a schema, and have rebuilt it to power a tool like GraphiQL) then we have very different validation needs. Assumptions made by execution like an object adhering to its implemented interfaces, or (in this case) names being well formed may not break assumptions of the tools being used - we're both paying a computation cost to perform this validation and we're forcing tools to be more strict than they might need to be. Should GraphiQL stop working or report errors if it's used against a server that has an invalid schema? If it should, how do we define that line? This diff is a great incremental step to easing the pain of this particular change, but it would be awesome to see a broader rethink of how schema validation fits into the whole story. |
| @@ -0,0 +1,139 @@ | |||
| /** | |||
| const error = new Error( | ||
| `Name "${name}" must not begin with "__", which is reserved by ` + | ||
| 'GraphQL introspection.' | ||
| 'GraphQL introspection. In a future release of graphql this will ' + |
There was a problem hiding this comment.
nit: graphql -> graphql-js?
There was a problem hiding this comment.
I went with graphql because that is the NPM package name. graphql-js is only the name of the GitHub repo.
|
Split off #719 to carry out discussion about the deeper changes to the validation model. |
A few things here:
console.errortoconsole.warn.