Skip to content

Output Formatting #679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Dec 28, 2018
+688 −16
Merged

Output Formatting #679

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
cc8957f
Add 'output formatting' markdown composition as working medium
gregsdennis Nov 18, 2018
2a4b28b
Add exceedingly large verbose output as separate file
gregsdennis Nov 18, 2018
6571b06
Add composition to spec XML
gregsdennis Nov 18, 2018
59f9ff0
updated per @Anthropic's suggestions :+1:
gregsdennis Nov 30, 2018
cf7a6e0
addressed most of @handrews's comments
gregsdennis Dec 5, 2018
fd1b00d
removed composition file; added output schema
gregsdennis Dec 6, 2018
df63510
clarified requirements around implementation support
gregsdennis Dec 7, 2018
06cd268
addressed some wording concerns
gregsdennis Dec 7, 2018
5504ffc
added comment for relative pointers in output formatting
gregsdennis Dec 10, 2018
32c624b
Updated language around format support requirements
gregsdennis Dec 18, 2018
fb312cc
Set support requirements per conversation in slack
gregsdennis Dec 21, 2018
02295dc
better wording for output
gregsdennis Dec 25, 2018
063fbe2
a bit more wordsmithing around output requirements
gregsdennis Dec 27, 2018
235c54a
Merge branch 'master' into draft-08-output-formatting
handrews Dec 28, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
489 changes: 473 additions & 16 deletions jsonschema-core.xml
View file
Original file line number Diff line number Diff line change
@@ -41,13 +41,25 @@
</address>
</author>

<author fullname="Ben Hutton" initials="B" surname="Hutton" role="editor">
<author fullname="Ben Hutton" initials="B" surname="Hutton" role="editor">
<organization>Wellcome Sanger Institute</organization>
<address>
<email>bh7@sanger.ac.uk</email>
</address>
</author>

<author fullname="Greg Dennis" initials="G" surname="Dennis">
<address>
<postal>
<street></street>
<city>Auckland</city>
<region></region>
<country>NZ</country>
</postal>
<email>gregsdennis@yahoo.com</email>
</address>
</author>

<date year="2018"/>
<workgroup>Internet Engineering Task Force</workgroup>
<keyword>JSON</keyword>
@@ -93,7 +105,8 @@
This specification defines JSON Schema core terminology and mechanisms, including
pointing to another JSON Schema by reference,
dereferencing a JSON Schema reference,
and specifying the vocabulary being used.
specifying the vocabulary being used,
and defining the expected output.
</t>
<t>
Other specifications define the vocabularies that perform assertions about validation,
@@ -610,6 +623,9 @@
exists during the evaluation of a schema, typically together
with an instance document. The outermost dynamic scope is the
root schema of the schema document in which processing begins.
The path from this root schema to any particular keyword (that
includes any "$ref" and "$recursiveRef" keywords that may have
been resolved) is considered the keyword's "validation path."
<cref>
Or should this be the schema object at which processing
begins, even if it is not a root? This has some implications
@@ -628,8 +644,8 @@
dynamic parent, rather than examining the local lexically enclosing parent.
</t>
<t>
The concept of dynamic scope is primarily used with "$recursiveRef"
and "$recursiveAnchor", and should be considered and advanced feature
The concept of dynamic scope is primarily used with "$recursiveRef",
"$recursiveAnchor", and should be considered an advanced feature
and used with caution when defining additional keywords.
</t>
</section>
@@ -1683,14 +1699,6 @@
</section>

<section title="Collecting Annotations">
<t>
<cref>
The exact structure and format of the information collected is TBD,
but will be defined before the next draft. Some details of this
section may change as a result, but the overall process is expected
to remain the same. See GitHub issue #396 to track progress.
</cref>
</t>
<t>
Annotations are collected by keywords that explicitly define
annotation-collecting behavior. Note that boolean schemas cannot
@@ -1706,12 +1714,13 @@
The instance location to which it is attached, as a JSON Pointer
</t>
<t>
The absolute schema location of the attaching keyword, as a URI
The schema location path, indicating how reference keywords
such as "$ref" were followed to reach the absolute schema location.
</t>
<t>
The schema location path, indicating how reference keywords
such as "$ref" were followed to reach the absolute schema location
<cref>The exact format of this path is TBD, again see issue #396</cref>
The absolute schema location of the attaching keyword, as a URI.
This MAY be omitted if it is the same as the schema location path
from above.
</t>
<t>
The attached value(s)
@@ -2379,6 +2388,454 @@
</section>
</section>

<section title="Output Formatting" anchor="output">
<t>
JSON Schema is defined to be platform-independent. As such, to increase compatibility
across platforms, implementations SHOULD conform to a standard validation output
format. This section describes the minimum requirements that consumers will need to
properly interpret validation results.
</t>

<section title="Format">
<t>
JSON Schema output is defined using the JSON Schema data instance model as described
in section 4.2.1. Implementations MAY deviate from this as supported by their
specific languages and platforms, however it is RECOMMENDED that the output be
convertible to the JSON format defined herein via serialization or other means.
</t>
</section>

<section title="Output Formats">
<t>
This specification defines four output formats. See the "Output Structure"
section for the requirements of each format.
<list>
<t>
Flag - A boolean which simply indicates the overall validation result
with no further details.
</t>
<t>
Basic - Provides validation information in a flat list structure.
</t>
<t>
Detailed - Provides validation information in a condensed hierarchical
structure based on the structure of the schema.
</t>
<t>
Verbose - Provides validation information in an uncondensed hierarchical
structure that matches the exact structure of the schema.
</t>
</list>
An implementation SHOULD provide at least the "flag", "basic", or "detailed"
format and MAY provide the "verbose" format. If it provides one or more of the
complex formats, it MUST also provide the "flag" format. Implementations SHOULD
specify in their documentation which formats they support.
</t>

</section>

<section title="Minimum Information">
<t>
Beyond the simplistic "flag" output, additional information is useful to aid in
debugging a schema or instance. Each sub-result SHOULD contain the information
contained within this section at a minimum.
</t>
<t>
A single object that contains all of these components is considered an
output unit.
</t>
<t>
Implementations MAY elect to provide additional information.
</t>

<section title="Keyword Relative Location">
<t>
The relative location of the validating keyword that follows the validation
path. The value MUST be expressed as a JSON Pointer, and it MUST include
any by-reference applicators such as "$ref" or "$recursiveRef".
</t>
<figure>
<artwork>
<![CDATA[
#/properties/minLength/$ref/minimum
]]>
</artwork>
</figure>
<t>
Note that this pointer may not be resolvable due to the inclusion of these
applicator keywords.
</t>
<t>
The JSON key for this information is "keywordLocation".
</t>
</section>

<section title="Keyword Absolute Location">
<t>
The absolute, dereferenced location of the validating keyword. The value MUST
be expressed as an absolute URI, and it MUST NOT include by-reference applicators
such as "$ref" or "$recursiveRef".
</t>
<figure>
<artwork>
<![CDATA[
http://json-schema.org/draft-08/schema#/$defs/nonNegativeInteger/minimum
]]>
</artwork>
</figure>
<t>
This information MAY be omitted only if either the relative location contains
no references or if the schema does not declare an absolute URI as its "$id".
</t>
<t>
The JSON key for this information is "absoluteKeywordLocation".
</t>
</section>

<section title="Instance Location">
<t>
The location of the JSON value within the instance being validated. The
value MUST be expressed as a JSON Pointer.
</t>
<t>
The JSON key for this information is "instanceLocation".
</t>
</section>

<section title="Error or Annotation">
<t>
The error or annotation that is produced by the validation.
</t>
<t>
For errors, the specific wording for the message is not defined by this
specification. Implementations will need to provide this.
</t>
<t>
The JSON key for failed validations is "error"; for successful validations
it is "annotation".
</t>
</section>

<section title="Nested Results">
<t>
For the two hierarchical structures, this property will hold nested errors
and annotations.
</t>
<t>
The JSON key for nested results in failed validations is "errors"; for
successful validations it is "annotations".
</t>
</section>

</section>

<section title="Output Structure">
<t>
The output MUST be an object containing a boolean property named "valid". When
additional information about the result is required, the output MUST also contain
"errors" or "annotations" as described below.
<list>
<t>
"valid" - a boolean value indicating the overall validation success or
failure
</t>
<t>
"errors" - the collection of errors or annotations produced by a failed
validation
</t>
<t>
"annotations" - the collection of errors or annotations produced by a
successful validation
</t>
</list>
For these examples, the following schema and instance will be used.
</t>
<figure>
<artwork>
<![CDATA[
{
"$id": "http://example.com/polygon#",
"$schema": "http://json-schema.org/draft-08/schema#",
"$defs": {
"point": {
"type": "object",
"properties": {
"x": { "type": "number" },
"y": { "type": "number" }
},
"additionalProperties": false,
"required": [ "x", "y" ]
}
},
"type": "array",
"items": { "$ref": "#/$defs/point" },
"minItems": 3
}
[
{
"x": 2.5,
"y": 1.3,
},
{
"x": 1,
"z": 6.7
}
]
]]>
</artwork>
</figure>
<t>
This instance will fail validation and produce errors, but it's trivial to deduce
examples for passing schemas that produce annotations.
</t>
<t>
Specifically, the errors it will produce are:
<list>
<t>
The second element in the "vertices" property is missing a "y" property.
</t>
<t>
The second element in the "vertices" property has a disallowed "z" property.
</t>
<t>
There are only two vertices, but three are required.
</t>
</list>
Note that neither the error message property nor its wording as depicted in these
examples is not a requirement of this specification. Implementations SHOULD craft
error messages tailored for their audience.
</t>

<section title="Flag">
<t>
In the simplest case, merely the boolean result for the "valid" valid property
needs to be fulfilled.
</t>
<figure>
<artwork>
<![CDATA[
{
"valid": false
}
]]>
</artwork>
</figure>
<t>
Because no errors or annotations are returned with this format, it is
RECOMMENDED that implementations use short-circuiting logic to return
failure or success as soon as the outcome can be determined. For example,
if an "anyOf" keyword contains five sub-schemas, and the second one
passes, there is no need to check the other three. The logic can simply
return with success.
</t>
</section>

<section title="Basic">
<t>
The "Basic" structure is a flat list of output units.
</t>
<figure>
<artwork>
<![CDATA[
{
"valid": false,
"errors": [
{
"keywordLocation": "#",
"instanceLocation": "#",
"error": "A subschema had errors."
},
{
"keywordLocation": "#/items/$ref",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point",
"instanceLocation": "#/1",
"error": "A subschema had errors."
},
{
"keywordLocation": "#/items/$ref/required",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/required",
"instanceLocation": "#/1",
"error": "Required property 'y' not found."
},
{
"keywordLocation": "#/items/$ref/additionalProperties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/additionalProperties",
"instanceLocation": "#/1/z",
"error": "Additional property 'z' found but was invalid."
},
{
"keywordLocation": "#/minItems",
"instanceLocation": "#",
"error": "Expected at least 3 items but found 2"
}
]
}
]]>
</artwork>
</figure>
</section>

<section title="Detailed">
<t>
The "Detailed" structure is based on the schema and can be more readable
for both humans and machines. Having the structure organized this way makes
associations between the errors more apparent. For example, the fact that
the missing "y" property and the extra "z" property both stem from the same
location in the instance is not immediately obvious in the "Basic" structure.
In a hierarchy, the correllation is more easily identified.
</t>
<t>
The following rules govern the construction of the results object:
<list>
<t>
All applicator keywords ("*Of", "$ref", "if"/"then"/"else", etc.) require
a node.
</t>
<t>
Nodes that have no children are removed.
</t>
<t>
Nodes that have a single child are replaced by the child.
</t>
</list>
Branch nodes do not require an error message or an annotation.
</t>
<figure>
<artwork>
<![CDATA[
{
"valid": false,
"keywordLocation": "#",
"instanceLocation": "#",
"errors": [
{
"valid": false,
"keywordLocation": "#/items/$ref",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point",
"instanceLocation": "#/1",
"errors": [
{
"valid": false,
"keywordLocation": "#/items/$ref/required",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/required",
"instanceLocation": "#/1",
"error": "Required property 'y' not found."
},
{
"valid": false,
"keywordLocation": "#/items/$ref/additionalProperties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/additionalProperties",
"instanceLocation": "#/1/z",
"error": "Additional property 'z' found but was invalid."
}
]
},
{
"valid": false,
"keywordLocation": "#/minItems",
"instanceLocation": "#",
"error": "Expected at least 3 items but found 2"
}
]
}
]]>
</artwork>
</figure>
</section>

<section title="Verbose">
<t>
The "Verbose" structure is a fully realized hierarchy that exactly matches
that of the schema. This structure has applications in form generation and
validation where the error's location is important.
</t>
<t>
The primary difference between this and the "Detailed" structure is that
all results are returned. This includes sub-schema validation results that
would otherwise be removed (e.g. annotations for failed validations,
successful validations inside a `not` keyword, etc.). Because of this, it
is RECOMMENDED that each node also carry a `valid` property to indicate the
validation result for that node.
</t>
<t>
Because this output structure can be quite large, a smaller example is given
here for brevity. The full output structure of the example above can be found
<eref target="standardized-output-verbose.json">here</eref>.
</t>
<figure>
<artwork>
<![CDATA[
// schema
{
"$id": "http://example.com/polygon#",
"$schema": "http://json-schema.org/draft-08/schema#",
"type": "object",
"properties": {
"validProp": true,
},
"additionalProperties": false
}
// instance
{
"validProp": 5,
"disallowedProp": "value"
}
// result
{
"valid": false,
"keywordLocation": "#",
"instanceLocation": "#",
"errors": [
{
"valid": true,
"keywordLocation": "#/type",
"instanceLocation": "#"
},
{
"valid": true,
"keywordLocation": "#/properties",
"instanceLocation": "#"
},
{
"valid": false,
"keywordLocation": "#/additionalProperties",
"instanceLocation": "#",
"errors": [
{
"valid": false,
"keywordLocation": "#/additionalProperties",
"instanceLocation": "#/disallowedProp",
"error": "Additional property 'disallowedProp' found but was invalid."
}
]
}
]
}
]]>
</artwork>
</figure>
</section>

<section title="Output validation schema">
<t>
For convenience, a JSON Schema has been provided to validate output generated
by implementations. It can be found <eref target="schema-output.json">here</eref>.
</t>
</section>

</section>

</section>

<section title="Usage for Hypermedia" anchor="hypermedia">

<t>
85 changes: 85 additions & 0 deletions schema-output.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
{
"$schema": "http://json-schema.org/draft-08/schema#",
"$id": "http://json-schema.org/draft-08/schema-output#",
"description": "A schema that validates the minimum requirements for validation output",
"$defs": {
"outputUnit":{
"properties": {
"valid": { "type": "boolean" },
"keywordLocation": {
"type": "string",
"format": "uri-reference"
},
"absoluteKeywordLocation": {
"type": "string",
"format": "uri"
},
"instanceLocation": {
"type": "string",
"format": "uri-reference"
},
"errors": {
"$ref": "#/$defs/outputUnitArray"
},
"annotations": {
"$ref": "#/$defs/outputUnitArray"
}
},
"required": [ "valid", "keywordLocation", "instanceLocation" ],
"allOf": [
{
"if": {
"properties": {
"valid": { "const": false }
}
},
"then": {
"required": [ "errors" ]
}
},
{
"if": {
"oneOf": [
{
"properties": {
"keywordLocation": {
"pattern": ".*/$ref/.*"
}
}
},
{
"properties": {
"keywordLocation": {
"pattern": ".*/$recursiveRef/.*"
}
}
}
]
},
"then": {
"required": [ "absoluteKeywordLocation" ]
}
}
]
},
"outputUnitArray": {
"type": "array",
"items": { "$ref": "#/$defs/outputUnit" }
},
"flag": {
"properties": {
"valid": { "type": "boolean" }
},
"required": [ "valid" ]
},
"basic": { "$ref": "#/outputUnit" },
"detailed": { "$ref": "#/outputUnit" },
"verbose": { "$ref": "#/outputUnit" }
},
"oneOf": [
{ "$ref": "#/$defs/flag" },
{ "$ref": "#/$defs/basic" },
{ "$ref": "#/$defs/detailed" },
{ "$ref": "#/$defs/verbose" }
]
}
130 changes: 130 additions & 0 deletions standardized-output-verbose.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
{
"valid": false,
"keywordLocation": "#",
"instanceLocation": "#",
"errors": [
{
"valid": true,
"keywordLocation": "#/definitions",
"instanceLocation": "#"
},
{
"valid": true,
"keywordLocation": "#/type",
"instanceLocation": "#"
},
{
"valid": false,
"keywordLocation": "#/items",
"instanceLocation": "#",
"errors": [
{
"valid": true,
"keywordLocation": "#/items/$ref",
"absoluteKeywordLocation":
"http://example.com/polygon#/items/$ref",
"instanceLocation": "#/0",
"annotations": [
{
"valid": true,
"keywordLocation": "#/items/$ref",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point",
"instanceLocation": "#/0",
"annotations": [
{
"valid": true,
"keywordLocation": "#/items/$ref/type",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/type",
"instanceLocation": "#/0"
},
{
"valid": true,
"keywordLocation": "#/items/$ref/properties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/properties",
"instanceLocation": "#/0"
},
{
"valid": true,
"keywordLocation": "#/items/$ref/required",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/required",
"instanceLocation": "#/0"
},
{
"valid": true,
"keywordLocation": "#/items/$ref/additionalProperties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/additionalProperties",
"instanceLocation": "#/0"
}
]
}
]
},
{
"valid": false,
"keywordLocation": "#/items/$ref",
"absoluteKeywordLocation":
"http://example.com/polygon#/items/$ref",
"instanceLocation": "#/1",
"errors": [
{
"valid": false,
"keywordLocation": "#/items/$ref",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point",
"instanceLocation": "#/1",
"errors": [
{
"valid": true,
"keywordLocation": "#/items/$ref/type",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/type",
"instanceLocation": "#/1"
},
{
"valid": true,
"keywordLocation": "#/items/$ref/properties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/properties",
"instanceLocation": "#/1"
},
{
"valid": false,
"keywordLocation": "#/items/$ref/required",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/required",
"instanceLocation": "#/1"
},
{
"valid": false,
"keywordLocation": "#/items/$ref/additionalProperties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/additionalProperties",
"instanceLocation": "#/1",
"errors": [
{
"valid": false,
"keywordLocation": "#/items/$ref/additionalProperties",
"absoluteKeywordLocation":
"http://example.com/polygon#/definitions/point/additionalProperties",
"instanceLocation": "#/1/z"
}
]
}
]
}
]
}
]
},
{
"valid": false,
"keywordLocation": "#/minItems",
"instanceLocation": "#"
}
]
}