Question: How to generate the schema and wiring at runtime?

[nielsbasjes]

Hi,

For a project where the schema is to be determined at the start of the application (it can change depending on configuration) I'm looking at generating the schema and all related things at startup.

I have put my efforts (as clean as possible) in this test project.

https://github.com/nielsbasjes/spring-gql-test/blob/main/src/main/java/nl/basjes/experiments/springgqltest/MyDynamicGraphQLApi.java

Works partially.

The schema that is generated is what I have in mind for this test.

Yet when running this query:

query {
  DoSomething(thing: "foo") {
    commit
    url
  }
}

I expect this

{
  "data": {
    "DoSomething": {
      "commit": "Something",
      "url": "Something"
    }
  }
}

but I get this:

{
  "data": {
    "DoSomething": null
  }
}

So as far as I can tell I'm doing something wrong with regards of the runtime wiring.

As I have trouble finding a working demo application that does this I'm asking here.

What am I doing wrong?
What have I overlooked?
Where can I find the documentation I misunderstood?

Thanks.

[saintcoder]

Perhaps this could be built as a feature to create schema dynamically at startup, based Controllers and entities? If yes, please don't forget the documentation of schema generation too. Also, not all entity properties need to be exposed to schema.

[nielsbasjes]

In my case I need to be able to generate the fields that are present in a specific structure, and the schema which must then be linked to a specific getter call at run time for each field.

To over simplify what I'm looking for:

• My code generates a Map<String, String> and at startup I know all possible keys that can occur during runtime.
• At startup I want to generate the schema that holds all possible fields.
• If someone requests field "Foo" I want to call .get("Foo") on the instance that holds the fields and return the appropriate value.

If you know all fields at the time you write the code this is trivially hardcoded.
In my usecase I do not know all fields yet I want the same ease of use for the client.

As you can see my latest experiment does generate the Schema itself, yet I have not been able to link these fields to code that provides the required value to the client.

[rstoyanchev]

Apologies for the slow response.

The schemaFactory hook you're using takes two inputs, the TypeDefinitionRegistry and RuntimeWiring and expects both to be used. However, in your sample RuntimeWiring is not used, and that means no DataFetcher registrations.

By default, we use graphql.schema.idl.SchemaGenerator to create the schema from TypeDefinitionRegistry and RuntimeWiring. What I don't know currently is how to create it from GraphQLObjectTypes and a RuntimeWiring. It's more of a GraphQL Java question to get some answers to.

[nielsbasjes]

I'm still learning how all of these parts work together.
I think I see the part about the RuntimeWiring not being used, yet I do not yet understand how to do it correctly.
I see that the core GraphQL and the Spring-GraphQL are mixed in this, so what is the right place to get a basic example that does it correctly?

[nielsbasjes]

After you mentioning me not doing anything with the RuntimeWiring I did some more experimenting.

I now have a thing that works on my machine. LINK
It only uses the schemaFactory to add both the schema and runtimewiring additions.
What I found is that all of the supplied input (typeDefinitionRegistry and runtimeWiring) are effectively immutable. This leads to really nasty code.

@Configuration(proxyBeanMethods = false)
public class MyDynamicGraphQLApi {

    private static final Logger LOG = LogManager.getLogger(MyDynamicGraphQLApi.class);

    @Bean
    GraphQlSourceBuilderCustomizer graphQlSourceBuilderCustomizer() {
        return builder -> {

            // New type
            GraphQLObjectType version = GraphQLObjectType
                .newObject()
                .name("Version")
                .description("The version info")
                .field(newFieldDefinition()
                    .name("commit")
                    .description("The commit hash")
                    .type(Scalars.GraphQLString)
                    .build())
                .field(newFieldDefinition()
                    .name("url")
                    .description("Where can we find it")
                    .type(Scalars.GraphQLString)
                    .build())
                .build();

            // New "function" to be put in Query
            GraphQLFieldDefinition getVersion = newFieldDefinition()
                .name("getVersion")
                .description("It should return the do something here")
                .argument(GraphQLArgument.newArgument().name("thing").type(Scalars.GraphQLString).build())
                .type(version)
                .build();


            // Wiring for the new type
            TypeRuntimeWiring typeRuntimeWiringVersion =
                TypeRuntimeWiring
                    .newTypeWiring("Version")
                    .dataFetcher("commit", testDataFetcher)
                    .dataFetcher("url", testDataFetcher)
                    .build();

            // Wiring for the new "function"
            TypeRuntimeWiring typeRuntimeWiringQuery =
                    TypeRuntimeWiring
                        .newTypeWiring("Query")
                        .dataFetcher("getVersion", getVersionDataFetcher)
                        .build();

            builder
                .schemaFactory(
                    (typeDefinitionRegistry, runtimeWiring) -> {
                        // NOTE: Spring-Graphql DEMANDS a schema.graphqls with a valid schema or it will not load...

                        // ---------------------------------
                        // Extending the entries in Query

                        // We get the existing Query from the typeDefinitionRegistry (defined in the schema.graphqls file).
                        ObjectTypeDefinition query = (ObjectTypeDefinition) typeDefinitionRegistry.getType("Query").orElseThrow();
                        NodeChildrenContainer namedChildren = query.getNamedChildren();
                        List<Node> fieldDefinitions = namedChildren.getChildren(CHILD_FIELD_DEFINITIONS);

                        // We add all our new "functions" (field Definitions) that need to be added to the Query
                        fieldDefinitions.add(convert(getVersion));

                        // Add them all as extra fields to the existing Query
                        ObjectTypeDefinition queryWithNewChildren = query.withNewChildren(namedChildren);

                        // We remove the old "Query" and replace it with the version that has more children.
                        typeDefinitionRegistry.remove(query);
                        typeDefinitionRegistry.add(queryWithNewChildren);

                        // -----------------------
                        // Add all additional types (outside of Query)
                        typeDefinitionRegistry.add(convert(version));

                        // -----------------------
                        // Add all additional wiring
                        // NASTY 3: There is no simple 'addType' on an existing instance of RuntimeWiring.
                        addType(runtimeWiring, typeRuntimeWiringQuery);
                        addType(runtimeWiring, typeRuntimeWiringVersion);

                        // Now we create the Schema.
                        return new SchemaGenerator().makeExecutableSchema(typeDefinitionRegistry, runtimeWiring);
                    }
                );
        };
    }


    static DataFetcher<?> getVersionDataFetcher = environment -> {
        String arguments = environment
            .getArguments()
            .entrySet()
            .stream()
            .map(entry -> "{ " + entry.getKey() + " = " + entry.getValue().toString() + " }")
            .collect(Collectors.joining(" | "));

        String result = "getVersion Fetch: %s(%s)".formatted(environment.getField(), arguments);

        LOG.info("{}", result);
        return result;
    };

    static DataFetcher<?> testDataFetcher = environment -> {
        String arguments = environment
            .getArguments()
            .entrySet()
            .stream()
            .map(entry -> "{ " + entry.getKey() + " = " + entry.getValue().toString() + " }")
            .collect(Collectors.joining(" | "));

        String result = "Fetch: %s(%s)".formatted(environment.getField().getName(), arguments);

        LOG.info("{}", result);
        return result;
    };


    private FieldDefinition convert(GraphQLFieldDefinition field) {
        // NASTY: So far the only way I have been able to find to this conversion is to
        //   - wrap it in a GraphQLObjectType
        //   - Print it
        //   - and parse that String
        GraphQLObjectType query = GraphQLObjectType
                .newObject()
                .name("DUMMY")
                .field(field)
                .build();

        String print = new SchemaPrinter().print(query);
        ObjectTypeDefinition dummy = (ObjectTypeDefinition)new SchemaParser().parse(print).getType("DUMMY").orElseThrow();
        return dummy.getFieldDefinitions().get(0);
    }

    private TypeDefinition convert(GraphQLObjectType objectType) {
        String print = new SchemaPrinter().print(objectType);
        return new SchemaParser().parse(print).getType(objectType.getName()).orElseThrow();
    }

    // Yes, I know NASTY HACK
    public void addType(RuntimeWiring runtimeWiring, TypeRuntimeWiring typeRuntimeWiring) {
        Map<String, Map<String, DataFetcher>> dataFetchers = runtimeWiring.getDataFetchers();

        String typeName = typeRuntimeWiring.getTypeName();
        Map<String, DataFetcher> typeDataFetchers = dataFetchers.computeIfAbsent(typeName, k -> new LinkedHashMap<>());
        typeDataFetchers.putAll(typeRuntimeWiring.getFieldDataFetchers());

        TypeResolver typeResolver = typeRuntimeWiring.getTypeResolver();
        if (typeResolver != null) {
            runtimeWiring.getTypeResolvers().put(typeName, typeResolver);
        }
    }

}

Although this works I really do not like it.

I'm looking forward to learn on how to get to this end result in a clean way.

[bbakerman]

Hi there. I am a maintainer on the graphql-java engine project. @rstoyanchev asked me to chime in here on possible approaches.

Looking at your example you dont seem to far off however you are using the the AST classes instead of the GraphQlXXX schema element classes.

SDL text such as type Query { foo : String } are parsed into AST classes. Which your example code is manipulating, in a way your are editing the input SDL text.

Rather than do this you should build the base GraphqlSchema first and then edit that.

@Bean
GraphQlSourceBuilderCustomizer graphQlSourceBuilderCustomizer() {
    return builder -> {
        builder
            .schemaFactory(
                (typeDefinitionRegistry, runtimeWiring) -> {
                   // Now we create the base Schema.
                    GraphQlSchema baseSchema =  new SchemaGenerator().makeExecutableSchema(typeDefinitionRegistry, runtimeWiring);
                    GraphQlSchema finalSchema =  editBaseSchema(baseSchema);
                }

The above builds a base schema based on SDL and runtime wiring and then this gets edited via another mechanism.

That other mechanism is graphql.schema.SchemaTransformer#transformSchema(graphql.schema.GraphQLSchema, graphql.schema.GraphQLTypeVisitor)

There is a more documentation on editing a schema here

The basic gist is that for runtime schema elements (as opposed to AST element you are working on in your linked code) you use the runtime graphql.schema.GraphQLXXXXType schema element classes and you put your data fetchers into a graphql.schema.GraphQLCodeRegistry.

graphql.schema.GraphQLCodeRegistry is analogous to RuntimeWiring used in SDL schema generation but simpler. Its is how graphql.schema.GraphQLSchema keeps track of data fetchers.

The graphql.schema.GraphQLSchema is an immutable directed acyclic graph (DAG) and as such it needs careful editing even to add fields because immutable parent objects much be changed to edit the schema.

The following is an example of editing (adding) new fields and types.

GraphQLTypeVisitorStub visitor = new GraphQLTypeVisitorStub() {
    @Override
    public TraversalControl visitGraphQLObjectType(GraphQLObjectType objectType, TraverserContext<GraphQLSchemaElement> context) {
        GraphQLCodeRegistry.Builder codeRegistry = context.getVarFromParents(GraphQLCodeRegistry.Builder.class);
        // we need to change __XXX introspection types to have directive extensions
        if (someConditionalLogic(objectType)) {
            GraphQLObjectType newObjectType = buildChangedObjectType(objectType, codeRegistry);
            return changeNode(context, newObjectType);
        }
        return CONTINUE;
    }

    private boolean someConditionalLogic(GraphQLObjectType objectType) {
        // up to you to decide what causes a change, perhaps a directive is on the element
        return objectType.hasDirective("specialDirective");
    }

    private GraphQLObjectType buildChangedObjectType(GraphQLObjectType objectType, GraphQLCodeRegistry.Builder codeRegistry) {
        GraphQLFieldDefinition newField = GraphQLFieldDefinition.newFieldDefinition()
                .name("newField").type(Scalars.GraphQLString).build();
        GraphQLObjectType newObjectType = objectType.transform(builder -> builder.field(newField));

        DataFetcher newDataFetcher = dataFetchingEnvironment -> {
            return "someValueForTheNewField";
        };
        FieldCoordinates coordinates = FieldCoordinates.coordinates(objectType.getName(), newField.getName());
        codeRegistry.dataFetcher(coordinates, newDataFetcher);
        return newObjectType;
    }
};
GraphQLSchema newSchema = SchemaTransformer.transformSchema(baseSchema, visitor);

(please forgive the code above - I don't have a compiler handy so its illustrative and not necessarily compilable.)

[nielsbasjes]

Thanks this is really helpful. I'm going to try to get this running at my end.

One thing I find confusing:

• You mention the GraphQLSchema is an immutable directed acyclic graph (DAG) ,
• the website says several times it is an immutable cyclic graph.

I would expect a DAG, is this a typo in the website?

[nielsbasjes]

@bbakerman @rstoyanchev Thanks for helping out!

With your help I was able to implement what I was looking for:
https://github.com/nielsbasjes/yauaa/blob/main/webapp/src/main/java/nl/basjes/parse/useragent/servlet/graphql/AnalysisResultSchemaInitializer.java

Looking now at the Spring-GraphQL API and this solution direction, I'm now thinking that perhaps the Spring-GraphQL API for customization should be different.

Looking at what I know now perhaps a better API is simply a way of providing an instance of a GraphQLTypeVisitor? In my project that covered everything I needed and was quite easy to work with.

And perhaps even allow for a system to have multiple of those? That way libraries can be created that you simply include as a dependency and they add packaged features to the GraphQL api of a system, thus allowing for a kind of compositing way of working.
[I do realize that in the case of having multiple the ordering of these things may become a relevant thing to think about.]

So thanks for helping out and since my question has been answered I'm closing this issue.

[rstoyanchev]

@nielsbasjes thanks for the feedback. I see what you've arrived at with the schemaFactory that applies visitors with SchemaTransformer, but that also has to create GraphQLSchema first which is not relevant to the transformation.

We actually do support registration of GraphQlTypeVisitor via GraphQlSource.Builder but those were updated some time ago in f0ce942 to use SchemaTraverser instead of SchemaTransformer with the idea that it performs better and it actually covers all of our internal cases. The commit message there even says we can separately add an option to accept visitors for transformation as well, and that's what we'll do in #536.