Add specifying @language to expanded form

[msporny]

In order to fully support language-map round-tripping, we need to be able to express that a particular piece of markup in expanded form was specified using a language map. So, in order to round-trip this language map:

{
  tags: {
    en: {
      '@id': 'http://example.com/tags/foo', 'label': ' Foo'}
    de: {
      '@id': 'http://example.com/tags/baz', 'label': ' Baz'}
    }
}

We need to support this in expanded form:

{
  'http://example.org/vocab#tags': [{
    '@language': 'en', '@id': 'http://example.com/tags/foo', 'label': ' Foo' 
  }, {
    '@language': 'de', '@id': 'http://example.com/tags/baz', 'label': ' Baz'
  }]
}

To get back to this in compacted form:

{
  tags: {
    en: {
      '@id': 'http://example.com/tags/foo', 'label': ' Foo'}
    de: {
      '@id': 'http://example.com/tags/baz', 'label': ' Baz'}
    }
}

[lanthaler]

Sorry Manu, but I fear your proposal wouldn't work in this form. We will run into problems with @value objects that are not language-tagged strings. For example an integer would need to get an @language as well, so do typed values (meaning that you can't distinguish any more between a typed value and a language-tagged literal without relying on precedence rules).

The other problem I see is that if we allow completely arbitrary values in language maps, one could even include a language-tagged string which is tagged with another language than the one defined by the language-map and consequently round-tripping would break.

The only possible solution I see right now is to add a "@context": { "@language": __ } to all expanded values which would just express the default language... but that would be very weird because the fundamental functionality of expansion is to remove contexts altogether.

Nitpick: In the example above, after expansion, the value of "label" should be language-tagged strings.

[gkellogg]

I actually think it can probably work. It involves allowing @language to be used within node definitions. When expanding, string values would be expanded to include the included @language, such as:

{
  'http://example.org/vocab#tags': [{
    '@language': 'en', '@id': 'http://example.com/tags/foo', 'label': {'@value': ' Foo', '@language': 'en'} 
  }, {
    '@language': 'de', '@id': 'http://example.com/tags/baz', 'label': {'@value': ' Baz', '@language': 'de'}
  }]
}

As long as expanding leads to expanded values (which it does), this should allow compacting by adding rules for a local @language, and keeping that when compacting recursive objects.

Adding a '@context" can't be done when expanding, obviously. Otherwise, I think it can probably work, or I'm not completely understanding @lanthaler's concerns.

[lanthaler]

I was concerned about documents like this:

{
  "lang-map": {
    "en": [
      { "@id": "http://example.com/tags/foo", "label": "Foo.en" },
      1,
      "Foo.en",
      { "@value": "Foo" },
      { "@value": "Foo.type", "@type": "someType" }
    ],
    "de": [
      { "@id": "http://example.com/tags/bar", "label": "Bar.de" },
      2,
      "Bar.de",
      { "@value": "Bar" },
      { "@value": "Bar.type", "@type": "someType" }
    ]
  }
}

After expansion this would become:

  "lang-map": [
    {
      "@language": "en",
      "@id": "http://example.com/tags/foo",
      "label": [ { "@value": "Foo.en", "@language": "en" } ]
    },
    { "@value": 1, ***"@language": "en"*** }  <-- language-tagged number (xsd:integer)
    { "@value": "Foo.en", "@language": "en" }
    { "@value": "Foo" },   <-- can't add "@language": "en"! 
    { "@value": "Foo.type", "@type": "someType", ***"@language": "en"*** }  <-- precedence!?
    {
      "@language": "de",
      "@id": "http://example.com/tags/bar",
      "label": [ { "@value": "Bar.de", "@language": "de" } ]
    },
    { "@value": 2, ***"@language": "de" }  <-- language-tagged number (xsd:integer)
    { "@value: "Bar.de", "@language": "de" },
    { "@value": "Bar" },  <-- can't add "@language": "de"!
    { "@value": "Bar.type", "@type": "someType", ***"@language": "de"*** } <-- precedence!?
  ]

[msporny]

Current options under consideration by the group:

• Ask Drupal to change the data model (non-starter),
• Adopt a 1-to-1 mapping between compact/expanded form for language maps, (adds complexity to syntax)
• Adopt a complex algorithm to reconstruct language maps from expanded form, (adds complexity to API, and may be non-deterministic)
• Model the data using BCP47 language code IRIs. (problematic from an RDF data model standpoint)

[lanthaler]

ACTION: Gregg to write up language-map spec text for the JSON-LD API.

[gkellogg]

There are two proposals for supporting round-tripping of language maps in JSON-LD. Language map syntax is supported using @container: @language in a context document as specified in issue #133. This issue relates to defining expansion for language map in the case where the value is not a simple string.

In the first proposal, expanded values (included typed values), node definitions and node references are given an "@language" key/value pair:

{
  "@context": {
    "name": {"@id": "http://schema.org/name", "@container": "@language"}
  },
  "@id": "http://buckingham.uk/queenie",
  "name": {
    "en": [
      "The Queen",
      88,
      {"@id": "http://example.com/the_queen"}
    ],
    "de": [
      "Die Königin",
      89,
      {"@id": "http://example.de/de_königin"}
    ]
  }
}

This expands to a single property, but all values have @language added, even node definitions and typed values. This allows compaction to reverse the operation and properly allocate values to language tags.

[{
  "@id": "http://buckingham.uk/queenie",
  "http://schema.org/name": [
    {"@value": "The Queen", "@language": "en"},
    {"@value": 88, "@language": "en"},
    {"@id": "http://example.com/the_queen", "@language": "en"},
    {"@value": "Die Königin", "@language": "de"},
    {"@value": 89, "@language": "de"},
    {"@id": "http://example.de/de_königin", "@language": "de"}
  ]
}]

A complication is that typed values (which may also include @type) now may also contain @language. This creates some complexity when selecting the term to apply to a typed value (i.e., one may also exist which is coerced to the datatype of the value), but as this is a corner-case which is an anti-pattern, it seems acceptable.

The alternative algorithm would use @langmap instead of @language for both node definitions and typed values:

[{
  "@id": "http://buckingham.uk/queenie",
  "http://schema.org/name": [
    {"@value": "The Queen", "@language": "en"},
    {"@value": 88, "@langmap": "en"},
    {"@id": "http://example.com/the_queen", "@langmap": "en"},
    {"@value": "Die Königin", "@language": "de"},
    {"@value": 89, "@langmap": "de"},
    {"@id": "http://example.de/de_königin", "@langmap": "de"}
  ]
}]

This removes the ambiguity, but adds a separate keyword principally to support anti-patterns.

The proposal is to implement the first solution.

PROPOSAL: When expanding language maps, add @language to all expanded values, even for data-typed values.

PROPOSAL: When compacting, allocate values to language map terms (if they exist) when they contain an @language key even if a term with datatype coercion matches the datatype of a value.

[lanthaler]

I'm fine with the proposals except the expanded/typed values corner-case, so this:

"name": {
    "en": [
      "A",
      1,
      true,

      { "@value": "B },
      { "@value": 2 },
      { "@value": false },
      { "@value": "C", "@type": "someType" },
      { "@value": 3, "@type": "xsd:integer" },
      { "@value": false, "@type": xsd:boolean" }
  ]
}

Should expand to

"http://schema.org/name": [
      { "@value": "A", "@language": "en" },
      { "@value": 1, "@language": "en" },
      { "@value": true, "@language": "en" },

      { "@value": "B },
      { "@value": 2 },
      { "@value": false },
      { "@value": "C", "@type": "someType" },
      { "@value": 3, "@type": "xsd:integer" },
      { "@value": false, "@type": xsd:boolean" }
}

This is completely in-line with how we treat values in expanded object form with normal properties. Name below won't be expanded to a language-tagged string:

{
  "@context": {
    "name": {"@id": "http://schema.org/name", "@language": "de"}
  },
  "name": { "@value": "plain literal" }
}

To be consistent, we should then also language-tag booleans and numbers for simple language-tagged properties.

PROPOSAL: When expanding language maps, add @language to all expanded values, _except for values that are already in expanded object form_.

[gkellogg]

Not adding @language to already expanded values is consistent with how we process things now, but it does mean that we can't round-trip this use. We need to carefully consider if we should continue this, and risk creating a round-tripping issue.

[lanthaler]

The same is true for simple language-tagged properties, values in expanded object form don't round-trip, and that's a feature™ :-)

{
  "@context": { "name": { "@id": "http://schema.org/name", "@language": "@en" } },
  "name": [
      "Language-tagged string",
      { "@value": "Plain literal" }
    ]
}

Otherwise there wouldn't be a way to express a plain literal in a language-tagged property. You would have to define another property or use the full IRI instead.

@linclark is it a requirement for Drupal that also things like

{
  "@context": { "name": { "@id": "http://schema.org/name", "@container": "@language" } },
  "name": {
     "en": [
      { "@value": "Plain literal" },
      { "@value": "Typed value", "@type": "someType" }
    ]
  }
}

(this is compact form, not expanded form) round-trip cleanly?

[msporny]

I'm trying to figure out where we are with this issue. We agreed that we want the simpler form of language maps in JSON-LD 1.0 - so, basically, we support only this form of language map in JSON-LD 1.0:

With this context:

{
"prop": {
"@id": "http://example.org/vocab#prop",
"@container": "@language"
}
}

and this compact form:

{
"prop": {
"en": "Foo",
"fr": "Fou",
"jp": "ふ"
}
}

would expand to:

{
"http://example.org/vocab#prop": [
{"@value": "Foo", "@language": "en"},
{"@value": "Fou", "@language": "fr"},
{"@value": "ふ", "@language": "jp"}
]
}

PROPOSAL 4: JSON-LD 1.0 will support simple language maps. When using a language map and expanding, if the term's language key's value is not a simple string, the rule for using the language map does not apply (all language-map values get dropped). When compacting, if all statements in the list are not simple @value/@language objects, then the term that defines the language map does not match (the statements are kept in expanded form).

[lanthaler]

Yes, this is basically what we came up with for #133:

RESOLVED: The values of the key-value pairs of a language map MUST be strings or arrays of strings. When expanded, the strings are tagged with the language specified by the key. When compacting, only language-tagged strings will match a term that has a "@container": "@language" mapping. Terms that have a "@container": "@language" mapping MUST NOT be type-coerced.

I would suggest we close this issue by simply referencing #133.

[lanthaler]

PROPOSAL 5: Close by referencing #133: no further actions required.

[niklasl]

+1 to PROPOSAL 5.

[msporny]

PROPOSAL 5: +1 (as long as we move the language over that I wrote above, it makes what we're doing clearer)

[lanthaler]

PROPOSAL 5: +1

[gkellogg]

PROPOSAL 5: +1

[tidoust]

PROPOSAL 5: +1

[lanthaler]

RESOLVED: Close ISSUE-159 by referencing ISSUE-133: no further actions required.