The JSON context is incomplete and has other issues

[stuartasutton]

@siuc-nate, the CEASN context file at http://credreg.net/ctdlasn/schema/context/json is quite incomplete resulting in incorrect outcomes with the CASS data. My inventory (please double check) shows the following needing definition in the context file, otherwise, we get a lot of URIs handled as strings as opposed to things:

TYPE @ID

alignFrom
alignTo
broadAlignment
complexityLevel
comprisedOf
creator
dateCopyrighted 
derivedFrom
exactAlignment
hasChild
hasTopChild
isPartOf
isVersionOf
majorAlignment
minorAlignment
narrowAlignment
prerequisiteAlignment
publisher
rightsHolder

###XSD:DATE

dateCreated
dateValidFrom
dateValidUntil

###TYPE INLANGUAGE

inLanguage

The context file should contain everything needed for JSON data both inside and outside the CER; so, it should be a compete context for the CTDL schema.

ALSO

1. Also, please take a look at issue Reuse of CTDL properties and concept schemes in CE-ASN data #495 because it impacts the context file; and

2. We need to change the weight property to xsd:string in the definition table that still has xsd:float. We need to be able to express things like "23%". Values for this property are not subject to mathematical manipulation.

[siuc-nate]

I haven't checked every property but the pattern seems to be that the range of those properties is either a Competency or a CompetencyFramework, rather than a URI. Currently (perhaps incorrectly) the code that generates the context file will leave these out so as to accommodate use of both an { "@id": "http://..." } object and a "pointer" object (as we have been loosely defining it). Obviously this is more relevant to the context of the Registry.

At one point we did include such properties in the context file, declared as being @id properties, but we changed our minds on it later on. I believe this was due to the content in the Registry referencing the same context file, and thus needing the flexibility of using either @id objects or references for the various properties without breaking in RDF validators. In other words, we wanted the records in the Registry to be valid when paired with the same context file that everyone else would be using. I believe this came up somewhere in the midst of this issue, but that one is more about language maps and rather lengthy.

[stuartasutton]

Nate a range of Competency or CompetencyFramework IS a URI.

Nate please precisely define "pointer object" (in the context of the Registry)...bnodes of the types constituting what you call top-level?

[siuc-nate]

Yes.

In other words, we have to allow both of these:

{
  "somePropertyThatPointsToATopLevelObject": { "@id": "http://......" }
}

{
  "somePropertyThatPointsToATopLevelObject": { "@type": "TopLevelType", "name": "My name", "...": "..." }
}

or, to put it another way:

{
  "somePropertyThatPointsToATopLevelObject": [
    { "@type": "TopLevelType", "name": "My name", "...": "..." },
    { "@id": "http://......" }
  ]
}

If we specify that somePropertyThatPointsToATopLevelObject is always a URI string then we can only do

{
  "somePropertyThatPointsToATopLevelObject": [
    "http://......"
  ]
}

and would have no way to indicate objects that don't have URIs. This applies outside of the Registry, though the only "real" use cases we have so far are Registry-centric. So we can't restrict the context to only allow URI strings for properties that point to classes of objects where such objects may not have URIs to point to without breaking the Registry.

[siuc-nate]

Per our 1/30/2018 conversation:

The solution to this is, in hindsight, quite obvious and I'm not sure how we missed it in the first go-around. Essentially we:

• Declare the properties which point to top-level things to have a {"type": "@id"} designation in the @context
• Use blank node IRIs as the @id property for all reference/pointer objects
• Update the code that generates the context accordingly
• Update the code that generates the JSON validation schema accordingly
• Update the code that generates the metadata records accordingly
• Update the code that consumes the metadata records accordingly
• Update the documentation accordingly

[siuc-nate]

@cwd-mparsons @stuartasutton @jkitchensSIUC this one is a big issue, similar to language maps in scale/impact, because it affects all pointer object implementations. So it impacts:

• Credential Registry profile of CTDL (and relevant JSON validation schema)
• Credential Registry profile of CTDL-ASN (I don't think we really use pointer objects in any implementation here)
• Credential Publisher (we use a lot of pointer objects - may also have implications for the editor itself?)
• Credential Finder (how it consumes data may be affected
• Documentation (we need to make sure to update this)

[stuartasutton]

Nate, for the sake of complete documentation, where you state "Use blank node IRIs as the @id property for all reference/pointer objects", you referred to bnode identifiers in the following convention using an underscore:

"@id": "_:N286badecb70b4ae6b86eb32e9ef97f88",
"@type": "ceterms:CredentialOrganization",

Thus the properties listed at the top of this message can be included in the @context as "@id".

This is a very, very significant issue and should be given the highest priority we can muster given all there is to do.

[siuc-nate]

Exactly. It will also impact the JSON validation in the sense that things will need to be sent to the registry wrapped in a @graph container (and consumed the same way, since the payloads will be signed).

That would likely have ended up happening anyway for competency frameworks, though, so perhaps it will at least help our records be/look more consistent.

[stuartasutton]

Yes, the @graph is a natural consequence.

One other small note, I simply used a uuid and not the ctid variant (ce-) in the example since the actual value doesn't matter downstream because when other parsers get hold of a graph with bnodes, they substitute their own favorite form for bnode identifier. The result is the same, but the nodeID is inevitably different.

[siuc-nate]

So we would effectively go from:

//Data in the referenced context file - note there is no definition currently for the targetAssessment property
"@context": {

}

//Published credential data
{
  "ceterms:name": "My Credential",
  "ceterms:requires": [
    { 
      "@type": "ceterms:ConditionProfile",
      "ceterms:targetAssessment": [
        { 
          "@id": "http://credentialengine.../123" 
        },
        {
          "@type": "ceterms:AssessmentProfile",
          "ceterms:name": "Referenced Assessment",
        }
      ]
    }
  ]
}

to:

//Data in the referenced context file
"@context": {
  "ceterms:targetAssessment": { "@type": "@id" }
}

//Published credential data - note the use of the @graph wrapper
"@graph": [
  {
    "ceterms:name": "My Credential",
    "ceterms:requires": [
      { 
        "@type": "ceterms:ConditionProfile",
        "ceterms:targetAssessment": [
          "http://credentialengine.../123",
          "_:ABC"
        ]
      }
    ]
  },
  {
    "@id": "_:ABC",
    "@type": "ceterms:AssessmentProfile",
    "ceterms:name": "Referenced Assessment",
   }
]

[siuc-nate]

Updated implementation steps:

1. Update the JSON Schema validation
2. Inform the Credential Registry team of the changes
3. Update the Publisher
4. Update the Finder
5. Update the Documentation

[siuc-nate]

This has been solved and will be handled by the implementation of #521.