Validation: Allow references to components in non-OAS files, through any path

[tedepstein]

KZOE is too restrictive in validating references. It correctly ensures that the references themselves are in contexts explicitly allowed by OpenAPI 2.0 and 3.0. But it incorrectly requires the referenced components to be reachable through an OAS-standard path.

The OpenAPI 2.0 spec shows examples of references to single-component files (or "fragment files"), and to components in non-OAS-standard paths:

Reference Object Example

{
	"$ref": "#/definitions/Pet"
}

$ref: '#/definitions/Pet'

Relative Schema File Example

{
  "$ref": "Pet.json"
}

$ref: 'Pet.yaml'

Relative Files With Embedded Schema Example

{
  "$ref": "definitions.json#/Pet"
}

$ref: 'definitions.yaml#/Pet'

The OpenAPI 3.0.1 spec has equivalent examples, and neither spec explicitly says that referenced components have to be in their designated locations, in a valid OpenAPI document.

So we should not impose these additional restrictions.

Example: Reference to single-component schema file

Given a Quotations_Schema.yaml file like this:

title: Quotations
description: received result for quotations
type: object
properties:
  zing:
    type: string
  zang:
    type: string

... the following OpenAPI 3.0 spec in the same folder should be valid:

---
openapi: "3.0.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /products:
    get:
      responses:  
        200:
          description: An array of products
          content:
            application/json:
              schema:
                $ref: "Quotations_Schema.yaml"

It fails to resolve the ref, showing a warning:

Invalid object reference, the referenced object is not of expected type.

Here's an OpenAPI 2.0 file that should also be valid in the same folder containing Quotations_Schema.yaml. It fails with the same warning.

---
swagger: "2.0"
info:
  version: 1.0.0
  title: TaxBlaster

paths:
  /taxFilings/{id}:
    get:
     responses:
        200:
          description: Successful response
          schema:
            $ref: "Quotations_Schema.yaml"

Example: Internal Reference to Non-Components Path

The following OpenAPI 3.0 document correctly resolves the parameter reference in /customers/parameters to the parameter object defined in /products/parameters:

---
openapi: "3.0.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /products:
    get:
      parameters:
      - name: page
        in: query
        description: page of results to return
        schema:
          type: integer
      responses:  
        200:
          description: An array of products
          content:
            application/json:
              schema:
                type: object
                properties:
                  productID:
                    type: string
                  productName:
                    type: string
    
  /customers:
    get:
      parameters:
      - $ref: "#/paths/~1products/get/parameters/0"
      responses:  
        200:
          description: An array of customers
          content:
            application/json:
              schema:
                type: object
                properties:
                  customerID:
                    type: string
                  customerName:
                    type: string

Here's the equivalent in Swagger-OpenAPI 2.0, which also works as expected:

---
swagger: "2.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /products:
    get:
      parameters:
      - name: page
        in: query
        description: page of results to return
        type: integer
      responses:  
        200:
          description: An array of products
          schema:
            type: object
            properties:
              productID:
                type: string
              productName:
                type: string
    
  /customers:
    get:
      parameters:
      - $ref: "#/paths/~1products/get/parameters/0"
      responses:  
        200:
          description: An array of customers
          schema:
            type: object
            properties:
              customerID:
                type: string
              customerName:
                type: string

Example: External reference to non-components path

The same also works for external references in OpenAPI 3.0:

ProductsPath.yaml

---
openapi: "3.0.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /products:
    get:
      parameters:
      - name: page
        in: query
        description: page of results to return
        schema:
          type: integer
      responses:  
        200:
          description: An array of products
          content:
            application/json:
              schema:
                type: object
                properties:
                  productID:
                    type: string
                  productName:
                    type: string
    

CustomersPath.yaml

---
openapi: "3.0.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /customers:
    get:
      parameters:
      - $ref: "ProductsPath.yaml#/paths/~1products/get/parameters/0"
      responses:  
        200:
          description: An array of customers
          content:
            application/json:
              schema:
                type: object
                properties:
                  customerID:
                    type: string
                  customerName:
                    type: string

Works in OAS2 as well:

ProductsPath_v2.yaml

---
swagger: "2.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /products:
    get:
      parameters:
      - name: page
        in: query
        description: page of results to return
        type: integer
      responses:  
        200:
          description: An array of products
          schema:
            type: object
            properties:
              productID:
                type: string
              productName:
                type: string

CustomersPath_v2.yaml

---
swagger: "2.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /customers:
    get:
      parameters:
      - $ref: "ProductsPath_v2.yaml#/paths/~1products/get/parameters/0"
      responses:  
        200:
          description: An array of customers
          schema:
            type: object
            properties:
              customerID:
                type: string
              customerName:
                type: string

Example: Reference to OAS-Standard Path in Invalid OAS Document

In both 2.0 and 3.0 versions, I can make the ProductsPath file invalid by removing the header, and the external reference to its parameter still works, as long as it's accessible through this path that would lead to a valid parameter if the target file were a valid OpenAPI document. I'll call this an "OAS-standard path."

ProductsPathNotOAS.yaml

paths:
  /products:
    get:
      parameters:
      - name: page
        in: query
        description: page of results to return
        schema:
          type: integer

CustomersPath.yaml

---
openapi: "3.0.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /customers:
    get:
      parameters:
      - $ref: "ProductsPathNotOAS.yaml#/paths/~1products/get/parameters/0"
      responses:  
        200:
          description: An array of customers
          content:
            application/json:
              schema:
                type: object
                properties:
                  customerID:
                    type: string
                  customerName:
                    type: string

ProductsPathNotOAS_v2.yaml

paths:
  /products:
    get:
      parameters:
      - name: page
        in: query
        description: page of results to return
        type: integer

CustomersPathNotOAS_v2.yaml

---
swagger: "2.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /customers:
    get:
      parameters:
      - $ref: "ProductsPathNotOAS_v2.yaml#/paths/~1products/get/parameters/0"
      responses:  
        200:
          description: An array of customers
          schema:
            type: object
            properties:
              customerID:
                type: string
              customerName:
                type: string

Example: Reference to Non-OAS-Standard Path

If I put the parameter reference in a completely non-OAS-standard path, i.e. putting the parameter definiton in some path like /foo/bar/parameter that is not allowed in OpenAPI at all, then the external reference fails.

ProductsPathReallyNotOAS.yaml

foo:
  bar:
    parameter:
      name: page
      in: query
      description: page of results to return
      schema:
        type: integer

CustomersPathReallyNotOAS.yaml

---
openapi: "3.0.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /customers:
    get:
      parameters:
      - $ref: "ProductsPathReallyNotOAS.yaml#/foo/bar/parameter"
      responses:  
        200:
          description: An array of customers
          content:
            application/json:
              schema:
                type: object
                properties:
                  customerID:
                    type: string
                  customerName:
                    type: string

ProductsPathReallyNotOAS_v2.yaml

foo:
  bar:
    parameter:
    - name: page
      in: query
      description: page of results to return
      type: integer

CustomersPathReallyNotOAS_v2.yaml

---
swagger: "2.0"
info:
  version: "1.0.0"
  title: BeamUp API

paths:
  /customers:
    get:
      parameters:
      - $ref: "ProductsPathReallyNotOAS_v2.yaml#/foo/bar/parameter"
      responses:  
        200:
          description: An array of customers
          schema:
            type: object
            properties:
              customerID:
                type: string
              customerName:
                type: string

[ghillairet]

@tedepstein I implemented a fix in 4ad3045 It uses a different approach for type reference validation. We had a built in mechanism but I found that I could launch validation on a subset of a JSON schema with the JSON schema library we are using. So I am using that now to validate referenced elements.

[ghillairet]

Screencast:

[tedepstein]

Looks promising, @ghillairet. Thanks for the fast turnaround!

[andylowry]

Implemented in #440

[tedepstein]

@ghillairet , there were some test failures after merging #440. @andylowry is investigating, but please check the current status when you're back online.

[ghillairet]

@tedepstein @andylowry Thanks for fixing this test. The fix looks good and It's working as expected.

[BenjamenMeyer]

@tedepstein I think this is a fair description of some of what I was doing; confirming per your request in #163