Bug: JSON Schema Not Respected ?

[ArEnSc]

What happened?

Given this JSON Schema

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "actions": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "action": {
                        "type": "string",
                        "enum": [
                            "go_to",
                            "pick_up_item",
                            "drop_item"
                        ]
                    }
                },
                "required": [
                    "action"
                ],
                "oneOf": [
                    {
                        "properties": {
                            "action": {
                                "const": "go_to"
                            },
                            "location": {
                                "type": "string",
                                "enum": [
                                    "mall",
                                    "backyard"
                                ]
                            }
                        },
                        "required": [
                            "location"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "lay_down"
                            },
                            "data": {
                                "type": "string",
                                "enum": [
                                    "None"
                                ]
                            }
                        },
                        "required": [
                            "data"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "pick_up_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "item"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "drop_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "item"
                        ]
                    }
                ]
            }
        }
    },
    "required": [
        "actions"
    ]
}

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are the smartest AI assistant in existence. You are an agent and are looking to decode sentences into so you can act out the commands. Convert the message inputs into actions in the form of json.<|eot_id|><|start_header_id|>user<|end_header_id|>

input message: Lets go to the mall, pick up a cake then drop it on the floor. Then take a coke, and drop it off in the backyard<|eot_id|><|start_header_id|>assistant<|end_header_id|>

output:

{"actions": [
{"location":"mall"}, 
{"item":"cake", "action":"pick_up_item"}, 
{"item":"cake", "action":"drop_item"} ,
{"item":"coke", "action":"pick_up_item"}, 
{"item":"coke", "action":"drop_item"}
] }

llama cpp server or llama cpp doesn't respect the JSON schema requiring an action parameter

more over there are a few more instances of this when you try for a longer schema that I can show that it seems to be not the model failing but the json schema

Name and Version

{"tid":"13408","timestamp":1717371771,"level":"INFO","function":"main","line":2918,"msg":"build info","build":2793,"commit":"8f8acc86"}

What operating system are you seeing the problem on?

Windows

Relevant log output

win_llamaserver\server.exe -m models/Meta-Llama-3-8B-Instruct-abliterated-v3_q6.gguf -ngl 9999 --ctx-size 8192 --parallel 4 --override-kv tokenizer.ggml.pre=str:llama3 -fa 
{"tid":"13408","timestamp":1717371771,"level":"INFO","function":"main","line":2918,"msg":"build info","build":2793,"commit":"8f8acc86"}
{"tid":"13408","timestamp":1717371771,"level":"INFO","function":"main","line":2925,"msg":"system info","n_threads":12,"n_threads_batch":-1,"total_threads":24,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | "}
llama_model_loader: loaded meta data with 22 key-value pairs and 291 tensors from models/Meta-Llama-3-8B-Instruct-abliterated-v3_q6.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = Meta-Llama-3-8B-Instruct-abliterated-v3
llama_model_loader: - kv   2:                          llama.block_count u32              = 32
llama_model_loader: - kv   3:                       llama.context_length u32              = 8192
llama_model_loader: - kv   4:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   7:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   8:                       llama.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                          general.file_type u32              = 18
llama_model_loader: - kv  11:                           llama.vocab_size u32              = 128256
llama_model_loader: - kv  12:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  14:                         tokenizer.ggml.pre str              = llama-bpe
llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,280147]  = ["─á ─á", "─á ─á─á─á", "─á─á ─á─á", "...
llama_model_loader: - kv  18:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 128001
llama_model_loader: - kv  20:                    tokenizer.chat_template str              = {% set loop_messages = messages %}{% ...
llama_model_loader: - kv  21:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q6_K:  226 tensors
validate_override: Using metadata override (  str) 'tokenizer.ggml.pre' = llama3
llm_load_vocab: special tokens definition check successful ( 256/128256 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 128256
llm_load_print_meta: n_merges         = 280147
llm_load_print_meta: n_ctx_train      = 8192
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 500000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 8192
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 8B
llm_load_print_meta: model ftype      = Q6_K
llm_load_print_meta: model params     = 8.03 B
llm_load_print_meta: model size       = 6.14 GiB (6.56 BPW)
llm_load_print_meta: general.name     = Meta-Llama-3-8B-Instruct-abliterated-v3
llm_load_print_meta: BOS token        = 128000 '<|begin_of_text|>'
llm_load_print_meta: EOS token        = 128001 '<|end_of_text|>'
llm_load_print_meta: LF token         = 128 'Ä'
llm_load_print_meta: EOT token        = 128009 '<|eot_id|>'
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
llm_load_tensors: ggml ctx size =    0.30 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU
llm_load_tensors:        CPU buffer size =   410.98 MiB
llm_load_tensors:      CUDA0 buffer size =  5871.99 MiB
.........................................................................................
llama_new_context_with_model: n_ctx      = 8192
llama_new_context_with_model: n_batch    = 2048
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 1
llama_new_context_with_model: freq_base  = 500000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      CUDA0 KV buffer size =  1024.00 MiB
llama_new_context_with_model: KV self size  = 1024.00 MiB, K (f16):  512.00 MiB, V (f16):  512.00 MiB
llama_new_context_with_model:  CUDA_Host  output buffer size =     2.45 MiB
llama_new_context_with_model:      CUDA0 compute buffer size =   258.50 MiB
llama_new_context_with_model:  CUDA_Host compute buffer size =    24.01 MiB
llama_new_context_with_model: graph nodes  = 903
llama_new_context_with_model: graph splits = 2
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"init","line":708,"msg":"initializing slots","n_slots":4}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"init","line":720,"msg":"new slot","id_slot":0,"n_ctx_slot":2048}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"init","line":720,"msg":"new slot","id_slot":1,"n_ctx_slot":2048}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"init","line":720,"msg":"new slot","id_slot":2,"n_ctx_slot":2048}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"init","line":720,"msg":"new slot","id_slot":3,"n_ctx_slot":2048}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"main","line":3015,"msg":"model loaded"}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"main","line":3040,"msg":"chat template","chat_example":"<|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHello<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHi there<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHow are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n","built_in":true}
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"main","line":3768,"msg":"HTTP server listening","hostname":"127.0.0.1","port":"8080","n_threads_http":"23"}  
{"tid":"13408","timestamp":1717371775,"level":"INFO","function":"update_slots","line":1807,"msg":"all slots are idle"}
{"tid":"13408","timestamp":1717371778,"level":"INFO","function":"launch_slot_with_task","line":1039,"msg":"slot is processing task","id_slot":3,"id_task":0}
{"tid":"13408","timestamp":1717371778,"level":"INFO","function":"update_slots","line":2091,"msg":"kv cache rm [p0, end)","id_slot":3,"id_task":0,"p0":0}
{"tid":"13408","timestamp":1717371786,"level":"INFO","function":"print_timings","line":320,"msg":"prompt eval time     =    1379.95 ms /    89 tokens (   15.51 ms per token,    64.50 tokens per second)","id_slot":3,"id_task":0,"t_prompt_processing":1379.95,"n_prompt_tokens_processed":89,"t_token":15.50505617977528,"n_tokens_second":64.49509040182615}
{"tid":"13408","timestamp":1717371786,"level":"INFO","function":"print_timings","line":336,"msg":"generation eval time =    6566.83 ms /    68 runs   (   96.57 ms per token,    10.36 tokens per second)","id_slot":3,"id_task":0,"t_token_generation":6566.826,"n_decoded":68,"t_token":96.5709705882353,"n_tokens_second":10.355078694029658}
{"tid":"13408","timestamp":1717371786,"level":"INFO","function":"print_timings","line":346,"msg":"          total time =    7946.78 ms","id_slot":3,"id_task":0,"t_prompt_processing":1379.95,"t_token_generation":6566.826,"t_total":7946.776}
{"tid":"13408","timestamp":1717371786,"level":"INFO","function":"update_slots","line":1789,"msg":"slot released","id_slot":3,"id_task":0,"n_ctx":8192,"n_past":156,"n_system_tokens":0,"n_cache_tokens":156,"truncated":false}
{"tid":"31532","timestamp":1717371786,"level":"INFO","function":"log_server_request","line":2869,"msg":"request","remote_addr":"127.0.0.1","remote_port":60337,"status":200,"method":"POST","path":"/completion","params":{}}
{"tid":"13408","timestamp":1717371786,"level":"INFO","function":"update_slots","line":1807,"msg":"all slots are idle"}
{"tid":"13408","timestamp":1717371786,"level":"INFO","function":"update_slots","line":1807,"msg":"all slots are idle"}

[ArEnSc]

Sampler settings here

   data = {
                "prompt": prompt,
                "n_predict": 256,
                "stream":stream,
                "cache_prompt":True,
                "id_slot": slot,
                "temperature": 0.0,
                "repeat_penalty": 1.25,
                "stop": ["<|eot_id|>"]
            }

[ArEnSc]

json schema:

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "actions": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "action": {
                        "type": "string",
                        "enum": [
                            "go_to",
                            "pick_up_item",
                            "drop_item",
                            "lay_down"
                        ]
                    }
                },
                "required": [
                    "action"
                ],
                "oneOf": [
                    {
                        "properties": {
                            "action": {
                                "const": "go_to"
                            },
                            "location": {
                                "type": "string",
                                "enum": [
                                    "mall",
                                    "backyard"
                                ]
                            }
                        },
                        "required": [
                            "location"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "lay_down"
                            },
                            "data": {
                                "type": "string",
                                "enum": [
                                    "None"
                                ]
                            }
                        },
                        "required": [
                            "data"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "pick_up_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "item"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "drop_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "item"
                        ]
                    }
                ]
            }
        }
    },
    "required": [
        "actions"
    ]
}

I made a minor addition to the enum and added "lay_down"

Input Prompt:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are the smartest AI assistant in existence. You are an agent and are looking to decode sentences into so you can act out the commands. Convert the message inputs into actions in the form of json.<|eot_id|><|start_header_id|>user<|end_header_id|>

input message: Lets go to the mall, pick up a cake then drop it on the floor. Then take a coke, and drop it off in the backyard  and lay down!<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{"actions": [{"location":"mall","action":"go_to"},{"item":"cake","action":"pick_up_item"}, {"location":"backyard", "action":"go_to"} ,{"item":"coke", "action":"pick_up_item"},{ "location":"backyard", "action":"go_to"}, {"item":"coke", "action": "drop_item"}, {"location": "backyard", "action": "go_to"},{"location": "backyard","action": "go_to"} ,{"location": "backyard", "action": "go_to"},{ "location": "backyard", "action": "go_to"}, {"location": "backyard", "action": "go_to"}, {"item": "cake", "action": "drop_item"},{"location":"backyard","action": "go_to"} ,{"location": "backyard", "action": "go_to"},{ "location": "backyard", "action": "go_to"}, { "location": "backyard", "action": "go_to"}, {"location": "backyard", "action": "go_to"},{"item":"cake","action": "drop_item"} ,{"location": "backyard", "action": "go_to
The inference function took 25.103541612625122 seconds to complete.

this one just kept going on forever, which seems odd, but its very likely has to do with just the schema bugs presented, even though we know that schemas may not terminate, I don't think that this is the case.

[ArEnSc]

More information
I use this url
self.url = "http://127.0.0.1:8080/completion"
and I do pass the schema off
data["json_schema"] = schema

[ArEnSc]

@ochafik
Maybe you have some insight here, I am guessing how this works is .... we take a json schema and convert it into a gbnf grammar? and that process is going wrong somewhere?
If so I guess to verify this as a bug

[HanClinto]

I'm still learning about schema-to-grammar conversion, but I figured I would have a go at confirming this report.

I am guessing how this works is .... we take a json schema and convert it into a gbnf grammar? and that process is going wrong somewhere? If so I guess to verify this as a bug

Yes, that's correct. There is a CPP program that does this (common/json-schema-to-grammar.cpp), as well as a Python program (examples/json_schema_to_grammar.py). The server uses the CPP route (so these reproduction steps won't be perfect), but we can easily call the Python program thusly:

python ./examples/json_schema_to_grammar.py ./grammars/issue7703.json

Which results in the output:

actions ::= "[" space (actions-item ("," space actions-item)*)? "]" space
actions-item ::= actions-item-0 | actions-item-1 | actions-item-2 | actions-item-3
actions-item-0 ::= "{" space actions-item-0-location-kv ( "," space ( actions-item-0-action-kv ) )? "}" space
actions-item-0-action ::= "\"go_to\""
actions-item-0-action-kv ::= "\"action\"" space ":" space actions-item-0-action
actions-item-0-location ::= "\"mall\"" | "\"backyard\""
actions-item-0-location-kv ::= "\"location\"" space ":" space actions-item-0-location
actions-item-1 ::= "{" space actions-item-1-data-kv ( "," space ( actions-item-1-action-kv ) )? "}" space
actions-item-1-action ::= "\"lay_down\""
actions-item-1-action-kv ::= "\"action\"" space ":" space actions-item-1-action
actions-item-1-data ::= "\"None\""
actions-item-1-data-kv ::= "\"data\"" space ":" space actions-item-1-data
actions-item-2 ::= "{" space actions-item-2-item-kv ( "," space ( actions-item-2-action-kv ) )? "}" space
actions-item-2-action ::= "\"pick_up_item\""
actions-item-2-action-kv ::= "\"action\"" space ":" space actions-item-2-action
actions-item-2-item ::= "\"vase\"" | "\"coke\"" | "\"cake\""
actions-item-2-item-kv ::= "\"item\"" space ":" space actions-item-2-item
actions-item-3 ::= "{" space actions-item-3-item-kv ( "," space ( actions-item-3-action-kv ) )? "}" space
actions-item-3-action ::= "\"drop_item\""
actions-item-3-action-kv ::= "\"action\"" space ":" space actions-item-3-action
actions-item-3-item ::= "\"vase\"" | "\"coke\"" | "\"cake\""
actions-item-3-item-kv ::= "\"item\"" space ":" space actions-item-3-item
actions-kv ::= "\"actions\"" space ":" space actions
root ::= "{" space actions-kv "}" space
space ::= " "?

I then tested this schema in the gbnf-validator program. Note that I had to change the final line of the grammar to space ::= [ \n]* to get it to handle the newlines present in your example output.

./gbnf-validator ./grammars/issue7703.gbnf ./grammars/issue7703_output.txt
Input string is valid according to the grammar.

If I plug your schema and the example output from your report into an online validator, then it tells me:

 Found 1 error(s)

Message:
    Required properties are missing from object: action.
Schema path:
    #/properties/actions/items/required

This confirms to me that the problem lies in the json-to-grammar conversion -- specifically in our handling of required properties (as you suspected).

Note that I also tested all of the above against #6640 (given that it has so many pending updates to schema-to-grammar conversion), but the same problems exist in that branch (indeed, the generated grammar seems to be identical between the branches).

Makes me wonder if required support is intended yet, or if it's a consciously-unsupported feature. As you noted, that's definitely a question for @ochafik

[HanClinto]

I've spent a few hours digging into this problem this evening, and have confirmed your problem. I started trying to wrap your bug in unit tests and uncovered some other issues along the way (see #7789 ). I have not yet added your specific test-case to the integration tests in #7790 , but I still intend to do so.

[ochafik]

@ArEnSc combining oneOf w/ properties isn't currently supported, llama.cpp just ignores the top-level action property and goes in to full oneOf mode

https://github.com/ggerganov/llama.cpp/blob/f5d7b268ec4bf8628aa6ccc9f6631d0230dde76f/common/json-schema-to-grammar.cpp#L629

(tbh it hadn't occurred to me one might want to combine the two, examples I saw weren't doing so, and I thought I'd seen the worst of it w/ tsconfig.json 😅)

This limitation should come w/ a warning or error (it's probably a bit hairy to support, and a bit niche), but in the meantime you can adjust your grammar as follows (you already had duplicated action properties in each of the oneOf alternatives, just make sure to make them required):

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "actions": {
            "type": "array",
            "items": {
                "oneOf": [
                    {
                        "properties": {
                            "action": {
                                "const": "go_to"
                            },
                            "location": {
                                "type": "string",
                                "enum": [
                                    "mall",
                                    "backyard"
                                ]
                            }
                        },
                        "required": [
                            "action", 
                            "location"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "lay_down"
                            },
                            "data": {
                                "type": "string",
                                "enum": [
                                    "None"
                                ]
                            }
                        },
                        "required": [
                            "action",
                            "data"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "pick_up_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "action",
                            "item"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "drop_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "action",
                            "item"
                        ]
                    }
                ]
            }
        }
    },
    "required": [
        "actions"
    ]
}

[ArEnSc]

@ArEnSc combining oneOf w/ properties isn't currently supported, llama.cpp just ignores the top-level action property and goes in to full oneOf mode

https://github.com/ggerganov/llama.cpp/blob/f5d7b268ec4bf8628aa6ccc9f6631d0230dde76f/common/json-schema-to-grammar.cpp#L629

(tbh it hadn't occurred to me one might want to combine the two, examples I saw weren't doing so, and I thought I'd seen the worst of it w/ tsconfig.json 😅)

This limitation should come w/ a warning or error (it's probably a bit hairy to support, and a bit niche), but in the meantime you can adjust your grammar as follows (you already had duplicated action properties in each of the oneOf alternatives, just make sure to make them required):

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "actions": {
            "type": "array",
            "items": {
                "oneOf": [
                    {
                        "properties": {
                            "action": {
                                "const": "go_to"
                            },
                            "location": {
                                "type": "string",
                                "enum": [
                                    "mall",
                                    "backyard"
                                ]
                            }
                        },
                        "required": [
                            "action", 
                            "location"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "lay_down"
                            },
                            "data": {
                                "type": "string",
                                "enum": [
                                    "None"
                                ]
                            }
                        },
                        "required": [
                            "action",
                            "data"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "pick_up_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "action",
                            "item"
                        ]
                    },
                    {
                        "properties": {
                            "action": {
                                "const": "drop_item"
                            },
                            "item": {
                                "type": "string",
                                "enum": [
                                    "vase",
                                    "coke",
                                    "cake"
                                ]
                            }
                        },
                        "required": [
                            "action",
                            "item"
                        ]
                    }
                ]
            }
        }
    },
    "required": [
        "actions"
    ]
}

yes this is a good solution thank you!

[ochafik]

Closing as this is now covered by grammars/README.md (#7841) :-)