ambiguous behaviour of additionalProperties when invalid

[ItsVeryWindy]

I was testing a schema and getting an additionalProperties error that didn't make much sense to me as the name of the property was in the schema.

So I put together a smaller schema to reproduce it.

{
    "type": "object",
    "properties": {
        "test": {
            "type": "string"
        }
    },
    "additionalProperties": false
}

I tried testing it against this piece of json.

{
  "test": 1
}

I tried it against a few of the online validators and depending on which implementation I tried the errors that I would get are different, sometimes I would just get the error about test not being a string, other times I would also get an error about no additional properties.

Reading http://json-schema.org/understanding-json-schema/reference/object.html#additional-properties

Specifically around properties whose names are not listed in the properties keyword or match any of the regular expressions in the patternProperties keyword implies that only the property name is important, not if it's valid or not.

I know that some of these implementations use annotations in order to determine if a property is additional or not, and those annotations will get dropped if any properties are invalid as referenced here, which is probably why I'm seeing what I am.

#939

Is there any expectations as to how additionalProperties is meant to work in these instances?

[karenetheridge]

That sounds like the issue raised in json-schema-org/community#57 and addressed for the next interim release in #1154.

[Relequestual]

I’m wondering if we read json-schema-org/community#57 incorrectly.

Schema objects that produce a false assertion result MUST NOT produce any annotation results, whether from their own keywords or from keywords in subschemas.
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.7.7.1.2

The annotation result of [the properties] keyword is the set of instance property names matched by this keyword.
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.10.3.2.1

In addition to possibly defining annotation results of their own, applicator keywords aggregate the annotations collected in their subschema(s) or referenced schema(s).
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.7.7.1.3

I read this as, the properties keyword makes its own annotations, which are not effected by the assertion results of the schema values of its object. The same is true for patternProperties.

[jviotti]

@Relequestual If I understand your point correctly, you are saying that properties should emit annotations that additionalProperties can use for its own behavior even if some of the subschemas do not match?

Consider the following schema:

{
    "properties": {
        "foo": true,
        "bar": false
    }
}

That schema has the following subschemas:

• #: The top schema
• #/properties/foo: Always produces a true assertion
• #/properties/bar: Always produces a false assertion

So at this point, based on the following:

Schema objects that produce a false assertion result MUST NOT produce any annotation results, whether from their own keywords or from keywords in subschemas.
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.7.7.1.2

We infer that #/properties/foo produces an annotation result, but #/properties/bar does not. Whether # produces an annotation result depends on the input. For example:

• {"foo":1}: The schema # validates successfully and the properties annotation is [ "foo" ]
• {}: The schema # validates successfully and the properties annotation is []
• {"bar":1}: The schema # DOES NOT validate successfully, so no annotations are emitted

Following the same logic:

• {"foo": 1, "bar":1}: The schema # DOES NOT validate successfully, so no annotations are emitted

So one way to read this:

In addition to possibly defining annotation results of their own, applicator keywords aggregate the annotations collected in their subschema(s) or referenced schema(s).
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.7.7.1.3

is the following: applicators may define extra annotations than the sum of their subschemas, but must emit nothing if the applicator does not validate successfully.

However, what about not? not is an applicator that by definition emits annotations DESPITE its subschema not validating successfully, which seems to violate the rule that says that "Schema objects that produce a false assertion result MUST NOT produce any annotation results, whether from their own keywords or from keywords in subschemas."

Therefore, an applicator seems to have a way to aggregate annotations from their subschemas, even if some fail. It follows that properties could have a behavior that aggregates annotations despite some of their subschemas failing.

However, there is still a key difference between not and properties: not succeeds when it's subschema fails, while properties only succeeds if ALL its subschemas are successful.

[jviotti]

I think oneOf is another interesting case to consider, as it is an applicator that emits annotations when one of its subschemas passes while the rest fail. anyOf also emits annotations when at least one of its subschemas pass while others may not.

[jviotti]

@Relequestual Actually, I think the issue reported here is valid.

I read this as, the properties keyword makes its own annotations, which are not effected by the assertion results of the schema values of its object.

The properties keyword makes its own annotations that are not affected by the assertion results of its subschemas, but properties itself expresses an assertion, and so properties can't emit annotations at all if its own assertion failed, independently of its sub assertions.

[Relequestual]

The properties keyword makes its own annotations that are not affected by the assertion results of its subschemas, but properties itself expresses an assertion, and so properties can't emit annotations at all if its own assertion failed, independently of its sub assertions.

properties is an applicator keyword. It does not produce any assertions. The assertions come from applying the schemas which are the values. properties just defines HOW to apply those schemas, if at all.

So, again I come back to...

Schema objects that produce a false assertion result MUST NOT produce any annotation results...

In the defintiion of properties...

"Validation succeeds if, for each name that appears in both the instance and as a name within this keyword's value, the child instance for that name successfully validates against the corresponding schema."

In contrast, here is allOf...

"An instance validates successfully against this keyword if it validates successfully against all schemas defined by this keyword's value."

It may seem like clutching at straws, and you COULD argue that properties combines the assertion results of the applications with logical AND, but I've never thought of it that way.

[jviotti]

properties is an applicator keyword. It does not produce any assertions. The assertions come from applying the schemas which are the values. properties just defines HOW to apply those schemas, if at all.

This is what I'm a bit confused about. The spec says:

JSON Schema keywords fall into several general behavior categories. Assertions validate that an instance satisfies constraints, producing a boolean result. Annotations attach information that applications may use in any way they see fit. Applicators apply subschemas to parts of the instance and combine their results.
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.7

The properties keyword is an applicator that combines the assertions produced by its subschemas. The result of properties is still a boolean result, which based on the definition above, means that properties is also an assertion?

Furthermore, the section that defines properties says this:

The value of "properties" MUST be an object. Each value of this object MUST be a valid JSON Schema.
Validation succeeds if, for each name that appears in both the instance and as a name within this keyword's value, the child instance for that name successfully validates against the corresponding schema.
The annotation result of this keyword is the set of instance property names matched by this keyword.
Omitting this keyword has the same assertion behavior as an empty object.
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.10.3.2.1

The above section mentions "validation succeeds if" (making it also sound like an assertion), and the final line says "Omitting this keyword has the same assertion behavior as an empty object" which implies that the presence of properties results in a different assertion, also making it sound like properties is an assertion?

I understand your point that properties just combines subschemas (in which case your premise about the current correctness of the spec holds), but I think there is a weird overlap between assertions and annotations that maybe is not well defined and leads to the confusion. There are many keywords that are "pure" annotations and do not interact with validation at all (like description) and "pure" assertions that evaluate to booleans such as type. However, keywords like properties and oneOf also evaluate to boolean results (so we can argue they perform validation/assertion) but still should not inherit the characteristics of other assertion keywords?

[gregsdennis]

properties is an applicator keyword. It does not produce any assertions. - @Relequestual

I think the categorization of a keyword into a single bucket is wrong. I address this in my opening comment on json-schema-org/community#63 (section on annotations).

[jviotti]

I think the spec explicitly mentions that keywords can fall within multiple categories:

Object properties that are applied to the instance are called keywords, or schema keywords. Broadly speaking, keywords fall into one of five categories:
...
Keywords may fall into multiple categories
https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.4.3.1

[jdesrosiers]

properties is an applicator keyword. It does not produce any assertions.

I'm sorry @Relequestual, but that doesn't make sense. Applicators assert on the result of applying one or more schemas. For properties, if all schemas are applied successfully, then the properties assertion is true. Without that, I don't see how your new interpretation makes sense.

Here's an example. Let me know if I understand your interpretation correctly.

{
  "type": "object",
  "properties": {
    "foo": { "type": "string" },
    "bar": { "type": "string" }
  },
  "additionalProperties": false
}

{
  "foo": 42,
  "bar": "hello"
}

This schema has three keywords at the top level: type, properties, and additionalProperties. type evaluates to true. properties applies the "foo" schema and finds it false. properties asserts false, but produces annotation ["bar"]. Assuming annotation-based evaluation is used, additionalProperties then determines that "foo" is additional based on the properties annotation and asserts false. This makes one passing keyword and two failing keywords, so the schema asserts false and doesn't pass along any of the annotations it's keyword's produced.

Under the current interpretation, both "foo" and "bar" would both be considered additional instead of just "foo" because properties wouldn't produce any annotations. The end result is that same, one passing keyword and two failing ones, but the additionalProperties would report a little differently.

Under the draft-07 interpretation, nothing is considered additional and additionalProperties passes. The spec says that both annotation-based and draft-07 evaluation are valid, but annotation-based evaluation (both interpretations) produces two errors while draft-07 evaluation produces one error. The end result is the same, but there are two possible output results. If that's something we care about, this part of the spec will need an update of some kind even if we adopt the new interpretation.

I still have to think a little more about how the new interpretation would affect unevaluatedProperties.