OAS3 Tag/template to define external-components URL

[gerardbosch]

Hi there,
I find myself using the power of $ref to reference external components in my API definitions, which is great as allows component reuse. But I find myself with this kind of code in my API definitions, where the links to external components are very long and hard to read:

  /foo/validate/{bar}:
    post:
      tags:
        - Foo
      summary: Validate Foo
      operationId: validateFoo
      parameters:
        # path
        - $ref: '#/components/parameters/bar'
      requestBody:
        $ref: '#/components/requestBodies/FooValidation'
      responses:
        '200':
          $ref: '#/components/responses/FooValidation'
        '400':
          $ref: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/BadRequest'
        '401':
          $ref: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/Unauthorized'
        '403':
          $ref: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/Forbidden'
        '500':
          $ref: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/InternalServerError'

I would really like to know if is there any way to define the base-path of that URLs in some kind of variable or YAML tag, so I could lead to $refs of this form:

$ref: '{apicomponents-errors}#/components/schemas/Something'

where I could define apicomponents-errors: http://... just once.

This is not only for better readability, but to allow update the versions all-in-once, as you can see from this example that the referenced components are versioned by its URL.

I have no idea if such kind of feature exists or can be implemented in OAS 3.

P.S. This does not only applies to schemas, but any kind of ref.

[handrews]

[EDIT: I misunderstood some stuff, see my next comment]

@gerardbosch $ref specifically takes a URI reference per RFC 3986, and unfortunately there is no concept of a base fragment against which you can resolve a partial fragment. Most fragment syntaxes do not have the sort of structure that JSON Pointer does, so the rule that when resolving references the fragment is always entirely replaced made a lot of sense when it was first created. If your fragments look like #foo, #bar, etc., there's no need for a base.

OpenAPI (or really JSON Schema as that is where this feature comes from) would need to come up with a completely different feature for this, because breaking RFC 3986 compatibility would be a huge problem.

In JSON Schema proper, this is handled by allowing $id to declare non-pointer fragments like #foo which can then be referenced from elsewhere in the document. That feature is not currently available in OpenAPI, though.

[gerardbosch]

@handrews Do you think this issue could be treated as a "Feature Request" in some way? I think it would be very beneficial to have clean $refs to allow proper component reuse, specially when defining collaborative APIs that can have several common components (schemas,...). And that would help having a clear view of the component versions in use as well as per my example.

What I pretend is a way of shortening and reusing the URL part to the .yaml file that contain those reusable components. Defining it once, and using it several times inside an API definition.

[handrews]

@gerardbosch Actually I'm now realizing that I misread your primary use case of a base URI in the sense of referencing another file/resource. Not referencing a different location in the same file with some sort of relative fragment.

$ref does take relative URI references, but you need to be able to establish the base URI. You would not be able to get down to just a fragment, for reasons I'm not going to get into at the moment, but you could easily do a relative path. Your filesytem layout seems like that might not get you much because of a deep nesting structure, but... if you want short relative references you should structure your directories accordingly.

There might be something here around an OpenAPI-specific variable substitution thing, which would ore closely match what you wrote- I'll leave that to someone else to respond to, I'm mostly just here for JSON Schema stuff 😃

[handrews]

Coming back to this- the base URI is determined by the Server Object's URLs. You would need to at least have the file name of the external file, plus the fragment, but that is a way that you can dramatically shrink the URLs in your $refs.

@gerardbosch does that answer your question? If you still have it- sorry for the delay.

[handrews]

@gerardbosch Many years later... we now have a proposal for OAS 3.2 that allows OAS documents to self-identify, which also sets the base URI.

So you could do something like this:

openapi: 3.2.0
self: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/openapi.yaml'
paths:
  /foo/validate/{bar}:
    post:
      responses:
        '200':
          $ref: '#/components/responses/FooValidation'
        '400':
          $ref: 'apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/BadRequest'

@gerardbosch if you are still interested in this, would this be sufficient for your needs?

[gerardbosch]

Hi @handrews, it has been a long time and right now I'm not sure as I'm not working in the same project currently.

But from your snippet above it's not very clear to me the location where the error is referenced as there's no http://....

So in this case, where is $ref: 'apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml read from?

And thanks for following up after so much time :)

[handrews]

@gerardbosch apologies again for the delayed reply, although shorter this time! My late Aug-Oct got rather hectic, and we weren't focusing on 3.2 yet which is where this would be addressed. We are now shifting to that release since 3.0.4 and 3.1.1 are finally out!

In this example, self sets a base URI of:

http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/openapi.yaml

When you see 'apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/BadRequest', what happens (per RFC3986) is that, since the URI-reference starts with a relative path (apicomponents-errors/...) and the base URI ends with a non-empty path segment (openapi.yaml), we replace that segment with the URI-reference, resulting in:

http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/BadRequest

So the combination of the new self field and the relative URI-reference $ref:

• self: http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/openapi.yaml in the OpenAPI Object
• $ref: 'apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/BadRequest' in place of a Response Object

is equivalent to:

• $ref: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/myproject/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml#/components/responses/BadRequest' in place of a Response Object

Does that help?

[gerardbosch]

Hi @handrews, thanks for taking time to check this after so long 🙏

Yes, this looks exactly what I was asking back in the day. I think it's a nice thing, although I had no time to investigate about the self field you mention. I understand that this new syntax will be only available when OAS 3.2 is released, right?

Anyway, thanks for the update! 🙂

[handrews]

@gerardbosch that is correct. I'm hoping we will get 3.2 out in the first half of 2025. Ideally by April, but hopefully at least by June. self (or possibly $self, there is some debate) will definitely be in 3.2.

[handrews]

This will be resolved by PR #4556.

[gerardbosch]

Hi @handrews, that's really nice that you take this in account after so many years 🙂. I have to admit that nowadays I'm not actively working with OpenApi YAML definitions as that was 6 years ago, hehe.

But I took a little time to revisit this and your explanation here, and if I understood well, I'm not very sure that the new $self feature would be very helpful for my use case.

Back in the day, what I was doing was:

• Microservices with polyrepo (one git repository per service).
• A separate dedicated git repository for each OpenApi definition (yaml document) 🢧 that repository was built into an API artifact by the CI (via OpenApi Generator for Java).

And to have a reusable set of API components, I had some other git repositories that only contained a YAML with some global reusable parts like an error message schema. Those yaml documents were just published to the corporate Artifactory as simple YAMLs (no code generation here), just with the goal to being referenced by the microservice API definition (second bullet above).

So in summary, my YAML definitions were published in URLs like this:

Service API:
http://repos.my-organization.com/artifactory/libs-release-local/foo/myproject/api/contract/api-payments/2.0/api-payments-2.0-api.yaml

Reusable components API:
http://repos.my-organization.com/artifactory/libs-release-local/foo/projectcodename/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml

---

So, if I was using $self on the api-payments as a base URL for $ref:

• Only a small part of the api-payments URL actually matches the one of apicomponents-errors (they were different repos, published at different paths);
• I would still need to repeat the version of the referenced component for each $ref I was using (see snippet at the start of the thread).

The idea was to be able to update them all at once in a single place, like a term or variable in the yaml, to be able to assign one (or several named base URLs). Then use those terms in $refs, for example:

    $ref: '${baseUrlApicomponentsErrors}#/components/responses/BadRequest
    ...
    $ref: '${baseUrlApicomponentsErrors}#/components/responses/Unauthorized
    ...
    $ref: '${baseUrlApicomponentsErrors}#/components/responses/InternalServerError

where baseUrlApicomponentsErrors was something like this (invented pseudocode below):

baseUrlApicomponentsErrors: 'http://repos.my-organization.foo/artifactory/libs-release-local/foo/projectcodename/api/contract/apicomponents-errors/1.0/apicomponents-errors-1.0-api.yaml'

---

Thinking nowadays about this, I think that using Git submodules for YAML dependencies would have been a neat way for that use case—would have allowed to deal with YAML dependencies as local files.

[handrews]

@gerardbosch thanks for replying even though you've moved on from this!

I do see your point here. $self might help more than you think beause it not only sets the base URI but sets the identity (hence URI instead of URL) of the document for use in $refs.

This means you can define a set of URIs independent from the location of the documents across different repos, and design the URIs to shorten the necessary relative references.

But it is true that it does not directly provide a templating system for multiple reference targets. I don't see us doing that, as implementors find the referencing system challenging enough as it is. The behavior of $self fits in with standard RFC3986 URI resolution behavior, and is analogous to $id in JSON Schema, so we're not asking implementors to do anything unusual (in a broad internet standards sense).

The other reason I don't see us doing this is that setting and using variables would be something to do in general, and not to fence off to a single aspect of the spec. Instead, we are likely to think through referencing more comprehensively for OAS 4.0 a.k.a. "Moonwalk."

But I definitely understand that designing an alternate set of URIs is not as flexible as setting and using variables.