Skip to content

The JSON context is incomplete and has other issues #508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stuartasutton opened this issue Jan 28, 2018 · 11 comments
Closed

The JSON context is incomplete and has other issues #508

stuartasutton opened this issue Jan 28, 2018 · 11 comments

Comments

@stuartasutton
Copy link
Contributor

stuartasutton commented Jan 28, 2018

@siuc-nate, the CEASN context file at http://credreg.net/ctdlasn/schema/context/json is quite incomplete resulting in incorrect outcomes with the CASS data. My inventory (please double check) shows the following needing definition in the context file, otherwise, we get a lot of URIs handled as strings as opposed to things:

TYPE @ID

alignFrom
alignTo
broadAlignment
complexityLevel
comprisedOf
creator
dateCopyrighted 
derivedFrom
exactAlignment
hasChild
hasTopChild
isPartOf
isVersionOf
majorAlignment
minorAlignment
narrowAlignment
prerequisiteAlignment
publisher
rightsHolder

###XSD:DATE

dateCreated
dateValidFrom
dateValidUntil

###TYPE INLANGUAGE

inLanguage

The context file should contain everything needed for JSON data both inside and outside the CER; so, it should be a compete context for the CTDL schema.

ALSO

  1. Also, please take a look at issue Reuse of CTDL properties and concept schemes in CE-ASN data #495 because it impacts the context file; and

  2. We need to change the weight property to xsd:string in the definition table that still has xsd:float. We need to be able to express things like "23%". Values for this property are not subject to mathematical manipulation.

@siuc-nate siuc-nate changed the title The ASN jason context is incomplete and has other issues The ASN JSON context is incomplete and has other issues Jan 29, 2018
@siuc-nate
Copy link
Contributor

siuc-nate commented Jan 29, 2018

I haven't checked every property but the pattern seems to be that the range of those properties is either a Competency or a CompetencyFramework, rather than a URI. Currently (perhaps incorrectly) the code that generates the context file will leave these out so as to accommodate use of both an { "@id": "http://..." } object and a "pointer" object (as we have been loosely defining it). Obviously this is more relevant to the context of the Registry.

At one point we did include such properties in the context file, declared as being @id properties, but we changed our minds on it later on. I believe this was due to the content in the Registry referencing the same context file, and thus needing the flexibility of using either @id objects or references for the various properties without breaking in RDF validators. In other words, we wanted the records in the Registry to be valid when paired with the same context file that everyone else would be using. I believe this came up somewhere in the midst of this issue, but that one is more about language maps and rather lengthy.

@stuartasutton
Copy link
Contributor Author

Nate a range of Competency or CompetencyFramework IS a URI.

Nate please precisely define "pointer object" (in the context of the Registry)...bnodes of the types constituting what you call top-level?

@siuc-nate
Copy link
Contributor

Yes.

In other words, we have to allow both of these:

{
  "somePropertyThatPointsToATopLevelObject": { "@id": "http://......" }
}
{
  "somePropertyThatPointsToATopLevelObject": { "@type": "TopLevelType", "name": "My name", "...": "..." }
}

or, to put it another way:

{
  "somePropertyThatPointsToATopLevelObject": [
    { "@type": "TopLevelType", "name": "My name", "...": "..." },
    { "@id": "http://......" }
  ]
}

If we specify that somePropertyThatPointsToATopLevelObject is always a URI string then we can only do

{
  "somePropertyThatPointsToATopLevelObject": [
    "http://......"
  ]
}

and would have no way to indicate objects that don't have URIs. This applies outside of the Registry, though the only "real" use cases we have so far are Registry-centric. So we can't restrict the context to only allow URI strings for properties that point to classes of objects where such objects may not have URIs to point to without breaking the Registry.

@siuc-nate
Copy link
Contributor

Per our 1/30/2018 conversation:

The solution to this is, in hindsight, quite obvious and I'm not sure how we missed it in the first go-around. Essentially we:

  • Declare the properties which point to top-level things to have a {"type": "@id"} designation in the @context
  • Use blank node IRIs as the @id property for all reference/pointer objects
  • Update the code that generates the context accordingly
  • Update the code that generates the JSON validation schema accordingly
  • Update the code that generates the metadata records accordingly
  • Update the code that consumes the metadata records accordingly
  • Update the documentation accordingly

@siuc-nate siuc-nate changed the title The ASN JSON context is incomplete and has other issues The JSON context is incomplete and has other issues Jan 30, 2018
@siuc-nate
Copy link
Contributor

@cwd-mparsons @stuartasutton @jkitchensSIUC this one is a big issue, similar to language maps in scale/impact, because it affects all pointer object implementations. So it impacts:

  • Credential Registry profile of CTDL (and relevant JSON validation schema)
  • Credential Registry profile of CTDL-ASN (I don't think we really use pointer objects in any implementation here)
  • Credential Publisher (we use a lot of pointer objects - may also have implications for the editor itself?)
  • Credential Finder (how it consumes data may be affected
  • Documentation (we need to make sure to update this)

@stuartasutton
Copy link
Contributor Author

Nate, for the sake of complete documentation, where you state "Use blank node IRIs as the @id property for all reference/pointer objects", you referred to bnode identifiers in the following convention using an underscore:

"@id": "_:N286badecb70b4ae6b86eb32e9ef97f88",
"@type": "ceterms:CredentialOrganization",

Thus the properties listed at the top of this message can be included in the @context as "@id".

This is a very, very significant issue and should be given the highest priority we can muster given all there is to do.

@siuc-nate
Copy link
Contributor

Exactly. It will also impact the JSON validation in the sense that things will need to be sent to the registry wrapped in a @graph container (and consumed the same way, since the payloads will be signed).

That would likely have ended up happening anyway for competency frameworks, though, so perhaps it will at least help our records be/look more consistent.

@stuartasutton
Copy link
Contributor Author

Yes, the @graph is a natural consequence.

One other small note, I simply used a uuid and not the ctid variant (ce-) in the example since the actual value doesn't matter downstream because when other parsers get hold of a graph with bnodes, they substitute their own favorite form for bnode identifier. The result is the same, but the nodeID is inevitably different.

@siuc-nate
Copy link
Contributor

So we would effectively go from:

//Data in the referenced context file - note there is no definition currently for the targetAssessment property
"@context": {

}

//Published credential data
{
  "ceterms:name": "My Credential",
  "ceterms:requires": [
    { 
      "@type": "ceterms:ConditionProfile",
      "ceterms:targetAssessment": [
        { 
          "@id": "http://credentialengine.../123" 
        },
        {
          "@type": "ceterms:AssessmentProfile",
          "ceterms:name": "Referenced Assessment",
        }
      ]
    }
  ]
}

to:

//Data in the referenced context file
"@context": {
  "ceterms:targetAssessment": { "@type": "@id" }
}

//Published credential data - note the use of the @graph wrapper
"@graph": [
  {
    "ceterms:name": "My Credential",
    "ceterms:requires": [
      { 
        "@type": "ceterms:ConditionProfile",
        "ceterms:targetAssessment": [
          "http://credentialengine.../123",
          "_:ABC"
        ]
      }
    ]
  },
  {
    "@id": "_:ABC",
    "@type": "ceterms:AssessmentProfile",
    "ceterms:name": "Referenced Assessment",
   }
]

@siuc-nate
Copy link
Contributor

Updated implementation steps:

  1. Update the JSON Schema validation
  2. Inform the Credential Registry team of the changes
  3. Update the Publisher
  4. Update the Finder
  5. Update the Documentation

@siuc-nate
Copy link
Contributor

This has been solved and will be handled by the implementation of #521.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment