Add support for emitting usage as a JSON schema #897

natebosch · 2025-06-13T19:26:27Z

Vendor a copy of a library defining a JSON schema API which can output
as a Map<String, Object?>.
Add a jsonSchema getter on ArgParser which is similar to usage
but outputs a schema instead of help text.

Clients can manually support a --json-help or similar argument in the
same way they support --help.

- Vendor a copy of a library defining a JSON schema API which can output as a `Map<String, Object?>`. - Add a `jsonSchema` getter on `ArgParser` which is similar to `usage` but outputs a schema instead of help text. Clients can manually support a `--json-help` or similar argument in the same way they support `--help`.

github-actions · 2025-06-13T19:28:47Z

PR Health

Breaking changes

⚠️

Package	Change	Current Version	New Version	Needed Version	Looking good?
args	Breaking	2.7.0	2.7.0	3.0.0 Got "2.7.0" expected >= "3.0.0" (breaking changes)	⚠️

This check can be disabled by tagging the PR with skip-breaking-check.

Changelog Entry ❗

Package	Changed Files
package:args	pkgs/args/lib/src/allow_anything_parser.dart pkgs/args/lib/src/arg_parser.dart pkgs/args/lib/src/json_schema.dart

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check.

Coverage

⚠️

File	Coverage
pkgs/args/lib/src/allow_anything_parser.dart	💔 72 % ⬇️ 4 %
pkgs/args/lib/src/arg_parser.dart	💚 95 % ⬆️ 1 %
pkgs/args/lib/src/json_schema.dart	💚 17 %

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package	Leaked API symbols

License Headers ✔️

// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

Files
no missing headers

All source files should start with a license header.

jakemac53 · 2025-06-16T14:23:28Z

pkgs/args/lib/src/json_schema.dart

+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+// Copied from https://github.com/dart-lang/ai/blob/failures-only/pkgs/dart_mcp/lib/src/api/tools.dart


Suggested change

// Copied from https://github.com/dart-lang/ai/blob/failures-only/pkgs/dart_mcp/lib/src/api/tools.dart

// Copied from https://github.com/dart-lang/ai/blob/main/pkgs/dart_mcp/lib/src/api/tools.dart

Maybe use an exact revision, or put a date on when it was copied.

A reference to a mutable file is informative, but really useful.
If the schema changes, it's good to know if we are behind or not.

Add strong language to always use the `run_tests` tool. Add a JSON schema for the CLI arguments to the test runner. The schema was dumped using dart-lang/core#897 Add a test for the conversion of boolean, string, and list of string arguments, as well as overriding the reporter argument.

lrhn · 2025-06-17T15:15:56Z

Has there been a request for this?

If we don't accept options being input as JSON, why are we giving a JSON schema for them?
(Do we accept them as JSON?)

lrhn · 2025-06-17T14:45:27Z

pkgs/args/lib/src/arg_parser.dart

+      }
+      if (extras.isNotEmpty) {
+        help = [
+          if (help != null) help,


Can use ?help now?

lrhn · 2025-06-17T14:47:21Z

pkgs/args/lib/src/json_schema.dart

+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+// Copied from https://github.com/dart-lang/ai/blob/failures-only/pkgs/dart_mcp/lib/src/api/tools.dart


Maybe use an exact revision, or put a date on when it was copied.

A reference to a mutable file is informative, but really useful.
If the schema changes, it's good to know if we are behind or not.

lrhn · 2025-06-17T14:49:15Z

pkgs/args/lib/src/json_schema.dart

+  string('string'),
+  num('number'),
+  int('integer'),
+  bool('boolean'),


Why different names and texts?

I can see that null can't work as a name, but the rest could just use the same name as the enum entry.

Then entry names are not the Dart types (not capitalized, and the Dart type for object is Map, not Object.)

lrhn · 2025-06-17T14:49:32Z

pkgs/args/lib/src/json_schema.dart

+  final String typeName;
+}
+
+/// A JSON Schema object defining the any kind of property.


lrhn · 2025-06-17T14:49:57Z

pkgs/args/lib/src/json_schema.dart

+/// See https://json-schema.org/understanding-json-schema/reference for the full
+/// specification.
+///
+/// **Note:** Only a subset of the json schema spec is supported by these types,


json -> JSON

lrhn · 2025-06-17T15:04:45Z

pkgs/args/lib/src/json_schema.dart

+    StringSchema? propertyNames,
+    int? minProperties,
+    int? maxProperties,
+  }) =>


(Could also be redirecting constructor : this.fromMap(...);. No big difference.)

lrhn · 2025-06-17T15:06:19Z

pkgs/args/lib/src/json_schema.dart

+        if (description != null) 'description': description,
+        if (properties != null) 'properties': properties,
+        if (patternProperties != null) 'patternProperties': patternProperties,
+        if (required != null) 'required': required,


With null-aware elements:

ObjectSchema.fromMap({ 'type': JsonType.object.typeName, 'title': ?title, 'description': ?description, 'properties': ?properties, 'patternProperties': ?patternProperties, 'required': ?required, 'additionalProperties': ?additionalProperties, 'unevaluatedProperties': ?unevaluatedProperties, 'propertyNames': ?propertyNames, 'minProperties': ?minProperties, 'maxProperties': ?maxProperties, });

lrhn · 2025-06-17T15:07:31Z

pkgs/args/lib/src/json_schema.dart

+  int? get maxProperties => _value['maxProperties'] as int?;
+}
+
+/// A JSON Schema definition for a String.


String -> `String`

Or "... for a [JsonType.string]."

lrhn · 2025-06-17T15:10:21Z

pkgs/args/lib/src/arg_parser.dart

@@ -392,4 +393,48 @@ class ArgParser {
  /// Finds the option whose name or alias matches [name], or `null` if no
  /// option has that name or alias.
  Option? findByNameOrAlias(String name) => options[_aliases[name] ?? name];
+
+  Map<String, Object?> get jsonSchema {


Documentation.

Why is this here? What can it be used for? By whom, and where?

lrhn · 2025-06-17T15:11:58Z

pkgs/args/lib/src/arg_parser.dart

+        ].join('\n');
+      }
+      final schema = switch (option.type) {
+        OptionType.flag => Schema.bool(


I'd just use BoolSchema directly, and probably drop the redirecting factories.

They're great for discovery, but this schema code here is completely internal, so we don't need that.

lrhn · 2025-06-17T15:18:19Z

pkgs/args/lib/src/json_schema.dart

+  const JsonType(this.typeName);
+
+  final String typeName;
+}


It looks like we're not using more that ~half of these, and we're not using the allOf-etc. properties at all.

Do we need a more complicated framework to generate a relatively simple JSON structure?

lrhn · 2025-06-17T15:32:39Z

pkgs/args/lib/src/arg_parser.dart

+      properties: properties,
+      required: required,
+    ).asMap();
+  }


So basically just:

var properties = <String, Object?>{}; var required = []; for (var option in _options.values) { if (option.hide) continue; var Option(:name, :type, :help) = option; final description = [?help, if (option.defaultsTo != null) 'defaults to "${option.defaultsTo}"' if (option.allowed?.isNotEmpty ?? false) 'allowed values: ${option.allowed?.join(', ')}' ].join('\n'); properties[name] = { ...switch (type) { OptionType.bool => const { 'type': 'boolean' }, OptionType.single => const { 'type': 'string' } , OptionType.multi => const { 'type': 'list', 'items': {'type': 'string'}, }, } if (description.isNotEmpty) 'description': description, }; if (option.mandatory) required.add(name); } return { 'type': 'object', 'properties': properties, if (required.isNotEmpty) 'required': required }; }

I'm not sure a complete JSONSchema implementation is worth it, especially one we didn't write ourselves, but is vendoring and now have to maintain anyway.

natebosch · 2025-06-17T17:51:46Z

Has there been a request for this?

I used this to create a schema for the test runner which we can use in our MCP server.
dart-lang/ai#169

I don't know if it makes sense to land this here since it might be breaking, but I at least wanted a reference implementation on a shared branch.

If we don't accept options being input as JSON, why are we giving a JSON schema for them?

I had suggested adding support for a JSON argument file to a few folks in a meeting and no one thought it was a good idea. I don't have strong feelings about it either way, it's pretty easy to convert. We could move the utility which does the conversion here if we want that part shared.

extension on Map<String, Object?> {
  Iterable<String> asCliArgs() sync* {
    for (final MapEntry(:key, :value) in entries) {
      if (value is List) {
        for (final element in value) {
          yield '--$key';
          yield element as String;
        }
        continue;
      }
      yield '--$key';
      if (value is bool) continue;
      yield value as String;
    }
  }
}

https://github.com/dart-lang/ai/pull/169/files#diff-c3420b6d2d3f69afb4dbb2c88882dc2a2d324dad52cf5853b3fd23b6c9436529R269

Add strong language to always use the `run_tests` tool. Add a JSON schema for the CLI arguments to the test runner. The schema was dumped using dart-lang/core#897 Add a test for the conversion of boolean, string, and list of string arguments, as well as overriding the reporter argument.

github-actions bot added the package:args label Jun 13, 2025

jakemac53 approved these changes Jun 16, 2025

View reviewed changes

natebosch mentioned this pull request Jun 17, 2025

Add JSON schema for test runner arguments dart-lang/ai#169

Merged

lrhn reviewed Jun 17, 2025

View reviewed changes

natebosch mentioned this pull request Jun 17, 2025

Add --json-help argument dart-lang/test#2508

Draft

	// Copied from https://github.com/dart-lang/ai/blob/failures-only/pkgs/dart_mcp/lib/src/api/tools.dart
	// Copied from https://github.com/dart-lang/ai/blob/main/pkgs/dart_mcp/lib/src/api/tools.dart

Add support for emitting usage as a JSON schema #897

Are you sure you want to change the base?

Add support for emitting usage as a JSON schema #897

Conversation

natebosch commented Jun 13, 2025

Uh oh!

github-actions bot commented Jun 13, 2025

PR Health

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lrhn commented Jun 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lrhn Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lrhn Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

natebosch commented Jun 17, 2025

Uh oh!

Uh oh!

lrhn Jun 17, 2025 •

edited

Loading

lrhn Jun 17, 2025 •

edited

Loading