feat(jsonschema): make JSON-LD specific properties required

[ttskch]

Q	A
Branch?	3.3
Tickets	N/A
License	MIT
Doc PR	N/A

This PR modifies SchemaFactory for JSON-LD to output JSON-LD specific properties such as @context @id @type as REQUIRED.

Below is a simple example.

#[ORM\Entity(repositoryClass: TodoRepository::class)]
#[ApiResource]
class Todo
{
    #[ORM\Id]
    #[ORM\GeneratedValue]
    #[ORM\Column]
    private ?int $id = null;

    #[ORM\Column(length: 255)]
    #[Assert\NotBlank]
    #[Assert\Length(max: 255)]
    private ?string $title = null;

    #[ORM\Column(type: Types::TEXT, nullable: true)]
    private ?string $description = null;

    #[ORM\Column]
    #[Assert\NotNull]
    private ?bool $done = null;

Currently, the following schema is generated for the above API resource.

JSON-LD specific properties such as @context @id @type should be output as required in the OpenAPI document. However, since the above schema is shared by both the request body and response body, if we set these properties to required, they will also be required in the request body.

In the first place, even though ApiPlatform\Hydra\JsonSchema\SchemaFactory does not add JSON-LD specific properties in the input context, the schemas for input and output are generated with the same key, so in the OpenAPI document, properties such as @context @id @type are output even in POST/PUT/PATCH operations. This is the fundamental problem.

See:

• core/src/OpenApi/Factory/OpenApiFactory.php

  Lines 258 to 260 in e867d07

  	$operationOutputSchema = $this->jsonSchemaFactory->buildSchema($resourceClass, $operationFormat, Schema::TYPE_OUTPUT, $operation, $schema, null, $forceSchemaCollection);
  	$operationOutputSchemas[$operationFormat] = $operationOutputSchema;
  	$this->appendSchemaDefinitions($schemas, $operationOutputSchema->getDefinitions());

• core/src/OpenApi/Factory/OpenApiFactory.php

  Lines 402 to 404 in e867d07

  	$operationInputSchema = $this->jsonSchemaFactory->buildSchema($resourceClass, $operationFormat, Schema::TYPE_INPUT, $operation, $schema, null, $forceSchemaCollection);
  	$operationInputSchemas[$operationFormat] = $operationInputSchema;
  	$this->appendSchemaDefinitions($schemas, $operationInputSchema->getDefinitions());

Therefore, in this PR, I first modified the schemas for input and output to be generated with different keys, so that JSON-LD specific properties would not be output during POST/PUT/PATCH operations.

And I made sure to always output JSON-LD specific properties such as @context @id @type as required (if those properties exist).

This ensures that JSON-LD specific properties are not required in the request body, and those properties are marked as required in the response body.

This has the advantage that the type information of API clients automatically generated by OpenAPI TypeScript etc. will be more precise.

  export interface operations {
      api_todos_post: {
          requestBody: {
-             "application/ld+json": components["schemas"]["Todo.jsonld"];
+             "application/ld+json": components["schemas"]["Todo.jsonld.input"];
          };
          responses: {
              201: {
                  content: {
-                     "application/ld+json": compoennts["schemas"]["Todo.jsonld"];
+                     "application/ld+json": compoennts["schemas"]["Todo.jsonld.output"];
                  };
              };
          };
      };
  };
  
  export interface components {
      schemas: {
+         "Todo.jsonld.input": {
+             readonly id?: number;
+             title: string;
+             description?: string | null;
+             done: boolean;
+         };
-         "Todo.jsonld": {
+         "Todo.jsonld.output": {
-             readonly "@context"?: string | {
+             readonly "@context": string | {
                  "@vocab": string;
                  /** @enum {string} */
                  hydra: "http://www.w3.org/ns/hydra/core#";
                  [key: string]: unknown;
              };
-             readonly "@id"?: string;
+             readonly "@id": string;
-             readonly "@type"?: string;
+             readonly "@type": string;
              readonly id?: number;
              title: string;
              description?: string | null;
              done: boolean;
          };
      };
  };

[ttskch]

@soyuka

Thank you for your review. I've accepted your suggestion and fix the code.

However, I noticed something and am starting to question this PR fix. I would like to hear your opinion.

Originally, the readOnly property should be automatically removed from the request body schema.

Swagger's OpenAPI Guide (based on OAS 3.0) states:

readOnly properties are included in responses but not in requests

--- https://swagger.io/docs/specification/data-models/data-types/#readonly-writeonly

And, OAS 3.0 states:

Declares the property as “read only”. This means that it MAY be sent as part of a response but SHOULD NOT be sent as part of the request. If the property is marked as readOnly being true and is in the required list, the required will take effect on the response only.

--- https://spec.openapis.org/oas/v3.0.0#fixed-fields-19

Thus, at least in OAS 3.0, the readOnly property should be automatically removed from the request body schema.

In fact, if you look closely at the output of the Swagger UI (the version we're using in API Platform 3.3), you'll see that the readOnly property has been removed from the request body schema.

However, in OAS 3.1, there is no such description regarding the readOnly property. In OAS 3.0, the readOnly field was an own extension of Schema Object, but in OAS 3.1, it's quoted from JSON Schema Validation, and as a result, the description of the readOnly field was removed from OAS.

https://spec.openapis.org/oas/v3.1.0#schema-object

This seems like a problem with OAS itself, but I couldn't find any issues or pull requests regarding this in the OAS repository.

If you believe that the readOnly property should be automatically removed from the request body schema even in OAS 3.1, all that API Platform's SchemaFactory should do is simply make the JSON-LD specific property required. Because those properties are readOnly originally.

About the implementation of major code generators

When automatically generating TypeScript code from the sample resources in this PR using OpenAPI Generator (typescript-axios), OpenAPI Generator (typescript-fetch), and OpenAPI TypeScript, the following code was generated.

OpenAPI Generator (typescript-axios)

The readOnly properties are included in the request body schema.

/**
 * 
 * @export
 * @interface TodoJsonld
 */
export interface TodoJsonld {
    /**
     * 
     * @type {TodoJsonldContext}
     * @memberof TodoJsonld
     */
    '@context'?: TodoJsonldContext;
    /**
     * 
     * @type {string}
     * @memberof TodoJsonld
     */
    '@id'?: string;
    /**
     * 
     * @type {string}
     * @memberof TodoJsonld
     */
    '@type'?: string;
    /**
     * 
     * @type {number}
     * @memberof TodoJsonld
     */
    'id'?: number;
    /**
     * 
     * @type {string}
     * @memberof TodoJsonld
     */
    'title': string;
    /**
     * 
     * @type {string}
     * @memberof TodoJsonld
     */
    'description'?: string | null;
    /**
     * 
     * @type {boolean}
     * @memberof TodoJsonld
     */
    'done': boolean;
}

export const TodoApiAxiosParamCreator = function (configuration?: Configuration) {
  return {
    // ...

    /**
     * Creates a Todo resource.
     * @summary Creates a Todo resource.
     * @param {TodoJsonld} todoJsonld The new Todo resource
     * @param {*} [options] Override http request option.
     * @throws {RequiredError}
     */
    apiTodosPost: async (todoJsonld: TodoJsonld, options: RawAxiosRequestConfig = {}): Promise<RequestArgs> => {
      // ...
    },
  }
}

OpenAPI Generator (typescript-fetch)

The readOnly properties are removed from the request body schema. (However, for JSON-LD specific properties it doesn't work fully due to this issue for now.)

export interface ApiTodosGetCollectionRequest {
  page?: number;
}

export interface ApiTodosIdDeleteRequest {
  id: string;
}

export interface ApiTodosIdGetRequest {
  id: string;
}

export interface ApiTodosIdPatchRequest {
  id: string;
  todo: Omit<Todo, 'id'>;
}

export interface ApiTodosIdPutRequest {
  id: string;
  todoJsonld: Omit<TodoJsonld, '@id'|'@type'|'id'>;
}

export interface ApiTodosPostRequest {
  todoJsonld: Omit<TodoJsonld, '@id'|'@type'|'id'>;
}

OpenAPI TypeScript

As mentioned in this PR description, the readOnly property is included in the request body schema.

The readOnly properties are included in the request body schema.

However, a solution is being considered to remove the readOnly property (and writeOnly property) from the request body schema.

What should API Platform do?

Although the issue of unclear handling of the readOnly property in OAS 3.1 remains, the basic idea is that the readOnly property should be removed from the request body schema.

If so, all API Platform's SchemaFactory needs to do is simply make the JSON-LD specific properties required.

However, doing this would create backwards compatibility issues for API Platform users currently using code generators such as OpenAPI Generator (typescript-axios) and OpenAPI TypeScript. This is because API clients generated by these "misimplemented" code generators will cause type errors if they do not include JSON-LD specific properties in the request body schema.

If we're going to strictly follow OAS, we should just make the JSON-LD specific properties required, but if we're concerned about practical backward compatibility issues, it may be safer to explicitly separate the schema between input and output, as this PR currently proposes.

What do you think about this issue?

Regards.

[soyuka]

what's this noRector attribute?

[ttskch]

Sorry, they are unnecessary comments. I had some copy-pasted code left from other test methods. I removed them via 9cc9c63.

See also #6408

[soyuka]

and also from the looks of it @id is sometimes used on request bodies (see #6225)

Can't we just say that these json-ld specific fields are optional? We need to make sure they can be used as part of the request body (as tried at #6402)

[ttskch]

@soyuka

I believe that @id, @context, and @type all must have values ​​in the output, is that correct?

If @id can be included in the request body as mentioned in #6225, shouldn't we separate the schema for input and output as originally proposed in this PR, and make @id, @context, and @type required in the output?

[ili101]

As mentioned I see that it's impossible to manage existing subobjects like in this example PHPUnit test (commit) (writableLink using @id) if you generate the client with OpenAPI Generator because there is no @id in input.

So if you want to make them mandatory it is necessary to separate at least the @id for input and output and make the input optional.
Or if you drop the idea to make them mandatory it's just needed to remove the readOnly from @id.

[ttskch]

@soyuka With the merge of #6451 into 3.3, it is now the correct specification that the request body includes @context @id @type.
Therefore, the current situation where they are not output in the JSON Schema of the request body is considered a bug, and I have changed the base branch of this PR from main to 3.3.

To organize the current situation, there are two points that we believe are correct:

• JSON-LD specific properties such as @context @id @type can be included in the request body. (See fix(jsonld): allow @id, @context and @type on denormalization 2 #6451)
• JSON-LD specific properties such as @context @id @type always have a value in the response body. (In other words, they are REQUIRED)

Therefore, this PR modifies SchemaFactory as follows:

• Separate the schema for input and output
• Make JSON-LD specific properties such as @context @id @type non-readOnly.
• Only in the schema for output, make JSON-LD specific properties such as @context @id @type required

This change should not cause any BC Breaks for users.

Regards.

[soyuka]

Thanks @ttskch !