RefValidator baseSchemaUrl Port Ignored

[jawaff]

Hello,

I'm not sure if this is an issue, but it might be. I just thought I'd bring it up to see if it really is an issue.

Anyway, I was attempting to define an "id" property in a test schema. That defines the base URL that "$ref" properties will build off of. Then I used a "$ref" property and expected to build off of the defined "id". Within the "$ref" I used the "/" prefix (you've defined this as a constant at RefValidator.REF_DOMAIN) in order to build off of the base domain path, but this is where the unexpected behavior comes into play.

If the "id" property is set to "http://localhost:9999", then I'd expect this "/location_schema.json" reference to resolve to "http://localhost:9999/location_schema.json". However, your library resolves this to "http://localhost/location_schema.json". This wouldn't be a problem if the port was 8080, but I'm using a nonstandard port in this situation. To almost get the behavior that I originally expected, I'd have to change my reference to "/:9999/location_schema.json". However that also doesn't work because your library resolves that reference to "http://localhost//:9999/location_schema.json".

This is the json schema that I'm referring to:

{
  "type" : "object",
  "id": "http://localhost:9999",
  "properties" : {
    "age" : {"$ref" : "#/definitions/Age"},
    "location" : {"$ref" : "/location_schema.json"}
  },
  "definitions" : {
    "Age" : {
      "type" : "number",
      "minimum" : 0
    }
  }
}

Links:
https://github.com/networknt/json-schema-validator/blob/master/src/main/java/com/networknt/schema/RefValidator.java
https://spacetelescope.github.io/understanding-json-schema/structuring.html#the-id-property

[jawaff]

If you agree that this is an issue, then I would gladly submit a pull request containing a fix and a test case.

[stevehu]

@Wafflemon Thanks for bringing it up. Yes. This is a problem as we are assuming the port number as default. It would be great if you could submit a pull request. To ensure that nobody can overwrite your changes in the future, it would be better to include one or two test cases if you can. Thanks.

[jawaff]

Hey @stevehu, I've been looking more into this issue and trying to fix some other issues that I could find in the RefValidator class as well. I ended up finding an existing test case that appears to be quietly failing. The problem lies within the src/test/resources/refRemote.json resource file. The "change resolution scope" schema at the end of the file is throwing an exception and causing this library to bypass validation for the associated tests.

Here's the exception that I'm getting:

21:19:52.693 [main] ERROR c.networknt.schema.JsonSchemaFactory - Failed to load json schema!
java.io.FileNotFoundException: http://localhost:1234/folderInteger.json
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1909)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
	at java.base/java.net.URL.openStream(URL.java:1117)
	at com.networknt.schema.url.StandardURLFetcher.fetch(StandardURLFetcher.java:14)
	at com.networknt.schema.JsonSchemaFactory.getSchema(JsonSchemaFactory.java:186)
	at com.networknt.schema.RefValidator.getRefSchema(RefValidator.java:66)
	at com.networknt.schema.RefValidator.<init>(RefValidator.java:44)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:488)
	at com.networknt.schema.ValidatorTypeCode.newValidator(ValidatorTypeCode.java:121)
	at com.networknt.schema.JsonMetaSchema.newValidator(JsonMetaSchema.java:240)
	at com.networknt.schema.ValidationContext.newValidator(ValidationContext.java:22)
	at com.networknt.schema.JsonSchema.read(JsonSchema.java:115)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:55)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:48)
	at com.networknt.schema.ItemsValidator.<init>(ItemsValidator.java:41)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:488)
	at com.networknt.schema.ValidatorTypeCode.newValidator(ValidatorTypeCode.java:121)
	at com.networknt.schema.JsonMetaSchema.newValidator(JsonMetaSchema.java:240)
	at com.networknt.schema.ValidationContext.newValidator(ValidationContext.java:22)
	at com.networknt.schema.JsonSchema.read(JsonSchema.java:115)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:55)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:48)
	at com.networknt.schema.ItemsValidator.<init>(ItemsValidator.java:41)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:488)
	at com.networknt.schema.ValidatorTypeCode.newValidator(ValidatorTypeCode.java:121)
	at com.networknt.schema.JsonMetaSchema.newValidator(JsonMetaSchema.java:240)
	at com.networknt.schema.ValidationContext.newValidator(ValidationContext.java:22)
	at com.networknt.schema.JsonSchema.read(JsonSchema.java:115)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:55)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:48)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:43)
	at com.networknt.schema.JsonSchemaFactory.newJsonSchema(JsonSchemaFactory.java:144)
	at com.networknt.schema.JsonSchemaFactory.getSchema(JsonSchemaFactory.java:208)
	at com.networknt.schema.JsonSchemaTest.runTestFile(JsonSchemaTest.java:77)
	at com.networknt.schema.JsonSchemaTest.testRefRemoteValidator(JsonSchemaTest.java:254)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Bypass validation due to invalid schema: java.io.FileNotFoundException: http://localhost:1234/folderInteger.json

What do you think was the intent of this test? I'm planning on fixing this, but I don't fully understand what the proper behavior was intended to be. It kind of looks like the "folderInteger.json" $ref field is supposed to resolve to "http://localhost:1234/folder/folderInteger.json". However, there is no src/test/resources/folder/folderInteger.json file in this repository, so I'm not 100% sure. Can you give me some insight into what you think should be the proper behavior?

[stevehu]

For an external $ref, it needs a web server to serve the JSON file. There is a built-in Undertow server in the test case to do that. Some of the updates might have broken the server which refers to some files that cannot be found.

[jawaff]

I understood that there's a web server that hosts the JSON files. All of the files in this repository under src/test/resources/tests are being hosted by that web server, from what I could tell. The test case that I was mentioning is referencing a file ("folderInteger.json") that isn't being hosted by the test web server at all.

I went back to look at the commits and it doesn't look like that file has ever existed in this repository. I'm going to check out the Json-Schema-Test-Suite and see if I can figure out what is going on. It looks like you may have just missed a few files when you were copying test schemas into this repository. I can probably figure out the intent of the test case by looking through the Json-Schema-Test-Suite.

Thanks for the response. I think I've got this issue mostly figured out.

[stevehu]

I have reviewed the code based on the information you provided. It looks like someone submitted a test case but forgot to copy the JSON file.