Directory queries allow for invalid values of "depth" without throwing an error

[Zipunrar]

Version of MarkLogic Java Client API

5.5.4

Version of MarkLogic Server

10.0-9.2

Java version

Java 11 OpenJDK

OS and version

ProductName: macOS
ProductVersion: 13.4
BuildVersion: 22F66

Input: Some code to illustrate the problem, preferably in a state that can be independently reproduced on our end

I found an potential bug / inconsistency for the usage of the "cts:directory-query" in XQuery vs. Java.
The official documentation for XQuery states, that the directory query will accept "1" or "infinity" as valid values for the depth parameter. However, in Java it is possible to build a directory query using a custom depth value, like 5 or 42 as integer.

Executing the following in XQuery will result in an exception:

cts:search(/, cts:directory-query("searchDirectoryDepthTest/", "2"))

XDMP-ARG: cts:directory-query("searchDirectoryDepthTest/", "2") -- arg2 is invalid

To demonstrate the behavior in Java, I created a small test:

@RunWith(SpringRunner.class)
@TestExecutionListeners (listeners = {DependencyInjectionTestExecutionListener.class, SearchDirectoryDepthTest.class})
@ContextConfiguration (classes = TestConfig.class)
public class SearchDirectoryDepthTest extends AbstractTestExecutionListener
{
    private static final List<String> TEST_DOCUMENTS = new LinkedList<>();

    @Autowired
    private DatabaseClient connection;

    @Override
    public void beforeTestClass(TestContext testContext) throws Exception
    {
        this.connection = testContext.getApplicationContext().getBean(DatabaseClient.class);

        final String uri0 = "searchDirectoryDepthTest/0";
        final String uri1 = "searchDirectoryDepthTest/foo/1";
        final String uri2 = "searchDirectoryDepthTest/foo/bar/2";
        final String uri3 = "searchDirectoryDepthTest/for/bar/baz/3";

        final TextDocumentManager documentManager = this.connection.newTextDocumentManager();

        final StringHandle data0 = new StringHandle("Level 0");
        final StringHandle data1 = new StringHandle("Level 1");
        final StringHandle data2 = new StringHandle("Level 2");
        final StringHandle data3 = new StringHandle("Level 3");

        documentManager.write(uri0, data0);
        documentManager.write(uri1, data1);
        documentManager.write(uri2, data2);
        documentManager.write(uri3, data3);

        TEST_DOCUMENTS.add(uri0);
        TEST_DOCUMENTS.add(uri1);
        TEST_DOCUMENTS.add(uri2);
        TEST_DOCUMENTS.add(uri3);
    }

    @Override
    public void afterTestClass(TestContext testContext) throws Exception
    {
        this.connection = testContext.getApplicationContext().getBean(DatabaseClient.class);

        TEST_DOCUMENTS.forEach(this.connection.newDocumentManager()::delete);
    }

    @Test
    public void testQueryDirectory_DepthOne()
    {
        final StructuredQueryBuilder builder = this.connection.newQueryManager().newStructuredQueryBuilder();

        final StructuredQueryDefinition qDef = builder.directory(1, "searchDirectoryDepthTest/");

        final XMLHandle actual = this.connection.newQueryManager().search(qDef, new XMLHandle());

        Assert.assertEquals("1", actual.get().attr("total"));
        Assert.assertEquals("searchDirectoryDepthTest/0", actual.get().find("*:result").attr("uri"));
    }

    @Test
    public void testQueryDirector_Infinity()
    {
        final StructuredQueryBuilder builder = this.connection.newQueryManager().newStructuredQueryBuilder();

        final StructuredQueryDefinition qDef = builder.directory(true, "searchDirectoryDepthTest/");

        final XMLHandle actual = this.connection.newQueryManager().search(qDef, new XMLHandle());

        Assert.assertEquals("4", actual.get().attr("total"));

        final Collection<String> actualUris = actual.get().find("*:result").attrs("uri");

        Assert.assertTrue(actualUris.contains("searchDirectoryDepthTest/0"));
        Assert.assertTrue(actualUris.contains("searchDirectoryDepthTest/foo/1"));
        Assert.assertTrue(actualUris.contains("searchDirectoryDepthTest/foo/bar/2"));
        Assert.assertTrue(actualUris.contains("searchDirectoryDepthTest/for/bar/baz/3"));
    }

    @Test
    public void testQueryDirectory_CustomDepth()
    {
        final StructuredQueryBuilder builder = this.connection.newQueryManager().newStructuredQueryBuilder();

        final StructuredQueryDefinition qDef = builder.directory(2, "searchDirectoryDepthTest/");

        final XMLHandle actual = this.connection.newQueryManager().search(qDef, new XMLHandle());

        Assert.assertEquals("2", actual.get().attr("total"));

        final Collection<String> actualUris = actual.get().find("*:result").attrs("uri");

        Assert.assertTrue(actualUris.contains("searchDirectoryDepthTest/foo/bar/2"));
        Assert.assertTrue(actualUris.contains("searchDirectoryDepthTest/for/bar/baz/3"));
    }

(the class XMLHandle, is a custom implementation of mine, but just wraps JDOM2 Document. However, I hope you get the point of the test, if not please let me know, and I can provide a version without it)

Actual output: What did you observe? What errors did you see? Can you attach the logs? (Java logs, MarkLogic logs)

The third test "testQueryDirectory_CustomDepth" using a custom depth value fails.

Expected output: What specifically did you expect to happen?

I expect that the third test case either throw an exception, to be consistent to the XQuery equivalent (which might also be solved by just removing the possibility to call this using a custom depth), or to except the custom depth correctly, and return the requested URIs.

[rjrudin]

Thanks @Zipunrar for the detailed report! I observed the same results as you did. I also tried out the serialized query by sending it directly to the /v1/search endpoint - example of that query:

<query xmlns="http://marklogic.com/appservices/search" xmlns:search="http://marklogic.com/appservices/search" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <directory-query depth="2"><uri>/a/</uri><infinite>false</infinite></directory-query>
</query>

And I get the same behavior - so it appears that the REST API is likely ignoring the depth attribute and instead only focusing on whether infinite is true or false, with false resulting in a depth of 1.

The Java Client may not eagerly throw an exception here in case a future version of MarkLogic were to support a value other than 1 for depth. So I'm thinking the right fix for now - without changing the interface - is to modifying the Javadocs for the directory method to make it clear that only a value of 1 is meaningful for depth. We'll look into a fix in the REST API as well.

[Zipunrar]

Ok, thank you for the quick response. In this case I will change our documentation too, and hope for a support of custom depths in the future.

[rjrudin]

Closed via 6.4.0.