[analyzer] Analyzer doesn analyze `part` of the library

[sgrekhov]

We have the following

// test.dart
library test_lib;
part 'part.dart';

var foo;

main() {
  foo = 1;
  print(bar);
}

// part.dart
part of test_lib;

final foo = "foo";
final bar = "bar";

Analyzer:
dart analyze test.dart reports no issues.
dart analyze part.dart reports The name 'foo' is already defined.
VM:
dart test.dart reports

part.dart: Error: 'foo' is already declared in this scope.
final foo = "foo";
      ^^^
test.dart: Context: Previous declaration of 'foo'.
var foo;
    ^^^
test.dart: Error: Can't assign to this.
  foo = 1;
      ^

Analyzer should report that part 'part.lib'; has some problems preventing execution

Tested on Dart SDK version: 3.4.0-edge.ffe2d1cf84bda4f4edcc9550817b7c033c76f8f8 (main) (Fri Jan 12 10:33:53 2024 +0100) on "linux_x64"

[eernstg]

(I haven't performed any experiments with file names, assuming it is all *.dart).

That's surprising! How would dart analyze part.dart know which library to use as the "owner" of part.dart if we also have this?:

// test2.dart
library test_lib;
part 'part.dart';

var foo2; // No name clashes.

main() {
  foo2 = 1;
  print(bar);
}

[sgrekhov]

Sorry for the typo. I mean part.dart of course. Updated

[bwilkerson]

Analyzer should report that part 'part.lib'; has some problems preventing execution

I'm assuming you mean that you think these diagnostics should be reported when analyzing the library that includes the part (test.dart in your example).

The documentation isn't clear on this point, but when you use dart analyze to analyze a single file it will report all of the diagnostics reported against code in that file. If the file happens to be a library with parts, the diagnostics reported against those parts will not be included in the output. It appears that it's working as intended.

How would dart analyze part.dart know which library to use as the "owner" of part.dart if we also have this?

It doesn't. It gets very confused. That's a big part of why we now allow, and encourage all users to use, URIs in the part of directive rather than library names. It prevents exactly this kind of confusion. I'm hoping that in a future version of Dart we can remove the ability for a part of directive to use a library name. I'd be even happier if we did so by removing the concept of parts, but that might be too much to hope for.

[sgrekhov]

The documentation isn't clear on this point, but when you use dart analyze to analyze a single file it will report all of the diagnostics reported against code in that file. If the file happens to be a library with parts, the diagnostics reported against those parts will not be included in the output. It appears that it's working as intended.

@eernstg do you agree?
@davidmorgan it affects https://dart-review.googlesource.com/c/sdk/+/345541.

[davidmorgan]

Thanks everyone! This looks complicated...

Per Sergey's comment (and the PR link), I'm coming at this from the point of view of the language tests, where I am changing the test runner to pick up error expectations from files beyond just the main test file.

compilation_t01.dart is a relevant test, it imports a broken part (duplicate definition) and so we can think about whether the error is reported in the part.

The command line used by the language tests does report the error:

DART_CONFIGURATION=ReleaseX64 out/ReleaseX64/dart-sdk/bin/dart out/ReleaseX64/gen/dartanalyzer.dart.snapshot -Dtest_runner.configuration=analyzer-asserts-linux --ignore-unrecognized-flags --packages=/usr/local/google/home/davidmorgan/git/dart-sdk/sdk/.dart_tool/package_config.json --format=json /usr/local/google/home/davidmorgan/git/dart-sdk/sdk/tests/co19/src/Language/Libraries_and_Scripts/Parts/compilation_t01.dart
{"version":1,"diagnostics":[{"code":"duplicate_definition","severity":"ERROR","type":"COMPILE_TIME_ERROR","location":{"file":"/usr/local/google/home/davidmorgan/git/dart-sdk/sdk/tests/co19/src/Language/Libraries_and_Scripts/Parts/part_0.dart","range":{"start":{"offset":291,"line":12,"column":7},"end":{"offset":294,"line":12,"column":10}}},"problemMessage":"The name 'foo' is already defined.","correctionMessage":"Try renaming one of the declarations.","contextMessages":[{"location":{"file":"/usr/local/google/home/davidmorgan/git/dart-sdk/sdk/tests/co19/src/Language/Libraries_and_Scripts/Parts/compilation_t01.dart","range":{"start":{"offset":824,"line":17,"column":451},"end":{"offset":827,"line":17,"column":454}}},"message":"The first definition of this name."}],"documentation":"https://dart.dev/diagnostics/duplicate_definition"}]}

and equivalent output with --format=machine or no --format arg. This command refuses to analyze the part by itself: "part_0.dart is a part and cannot be analyzed. Please pass in a library that contains this part."

Then, dart analyze is different, as Brian said "the diagnostics reported against those parts will not be included in the output":

dart analyze dart analyze tests/co19/src/Language/Libraries_and_Scripts/Parts/compilation_t01.dart --> no errors reported

Where it gets interesting is

dart analyze tests/co19/src/Language/Libraries_and_Scripts/Parts/part_0.dart

which does find and report the error, despite there being no single owner library. Playing around a bit it looks like it takes the first alphabetical owner library and ignores the rest; so if I add a duplicate definition to compilation_t04.dart, an alternative owner library, then it is not reported; unless I make compilation_t01.dart not an owner library, then the error I introduce in compilation_t04.dart is reported.

Aside: amidst all this complexity there is a trivial bug affecting the output, which is that the JSON gets the file of the context message right but the human-readable version does not:

"contextMessages":[{"message":"The first definition of this name.","location":{"file":"/usr/local/google/home/davidmorgan/git/dart-sdk/sdk/tests/co19/src/Language/Libraries_and_Scripts/Parts/compilation_t04.dart","offset":856,"length":3,"startLine":17,"startColumn":483,"endLine":17,"endColumn":486}}]
  error • part_0.dart:16:5 • The name 'bar' is already defined. Try renaming one of the declarations. • duplicate_definition
           - The first definition of this name at part_0.dart:17:483.

Okay, now some thoughts on what we should actually do here ;)

The language tests are not testing dart analyze, they're testing a different entrypoint. There is a difference in how part files are handled. As far as I can see the current way is better for the language tests: it reports more errors, so we can assert on more errors (once my test runner change lands); and it works always from the library, so there is no confusion about which file owns the part.

Then I think there are three things we need to do:

• dart analyze output isn't the right way to think about the language tests as it's not equivalent to what the tests do, we should switch to a different command. (Is dart dartanalyzer.dart.snapshot the only/best way?)
• Decide if there is any issue with the current "dart analyze" behaviour or if it's working as intended / will not fix.
• If desired, file an issue / follow up on the wrong human-readable output issue noted above.

How does that sound, please? :)

[sgrekhov]

My testing also confirms all of the above. If we preserve the current behaviour of the tools, then, I think, with the updated test runner the part tests should be rewritten as:

// test1.dart
import 'part1.dart`;

main() {...}

// part1.dart
library some_lib;

  Line of code containing error;
//^
// [analyzer] unspecified
// [cfe] unspecified

// test2.dart
library test_lib;
part 'part.dart';

main() {
...
}

// part.dart
part of test_lib;

  Line of code containing error;
//^
// [cfe] unspecified

For test1.dart the test runner expects errors in both CFE and analyzer, but for test2.dart tests should expect error in CFE only because analyzer doesn't 'dive' into files specified in part directive.

[davidmorgan]

It's true that dart analyze run on the test file on the command line doesn't report anything for the part file, but the analyzer used in other ways (run on the whole folder, running in the IDE, run via the different command line I gave) does report something; so I think it's useful to test it in the language tests, as it should cover those ways of using the analyzer.

Maybe this part of the discussion would better belong on the test_runner feature request, but we seem to have mostly the same people anyway :)

[davidmorgan]

@eernstg @bwilkerson @munificent not sure who is the right person / people to decide on what the language tests do here? Input appreciated please :) thanks!

[bwilkerson]

... not sure who is the right person / people to decide on what the language tests do here?

I'm sure it's "people", not just "person". And I think I'm the right representative from the analyzer team. But I agree we need someone from the language team and maybe someone from the infrastructure team, depending on how big a change we might want there.

The command line used by the language tests does report the error ...

The language tests are currently using the dartanalyzer tool. There is a strong desire (from the analyzer team at least) to remove that tool so that we have less code to maintain. The biggest difference I was previously aware of between that tool and the newer dart analyze tool is that dartanalyzer supports a mode in which the test runner can feed it files to analyze over stdin rather than on the command line. We know that we need to add that support to dart analyze (because dropping it would increase the time to run tests by far too much), but haven't been able to make the time to do so yet.

It's possible that we need to support reporting diagnostics from part files as part of the same support (that is, we could consider reporting diagnostics from part files only when in "test runner" mode).

Decide if there is any issue with the current "dart analyze" behaviour or if it's working as intended / will not fix.

I think this needs to take into consideration that the answer might differ depending on whether we're asking about the behavior for users or the behavior for the test runner.

The behavior isn't consistent with how the tests are currently written, so we either need to change

• the behavior of dart analyze so that diagnostics from parts are reported when analyzing the library, or
• the way the tests are written so that we don't include parts in multiple libraries.

I know that using the same part file from multiple libraries is convenient because it means fewer duplicated files, making both the writing and maintaining of tests easier. But there isn't, as far as I'm aware, any valid language semantics for this kind of sharing, which means that the analysis server can't be used when editing these files. If we'd like people to be able to open the test directory (or some subset thereof) in an IDE, then I think the current test structure is going to be an issue. I don't want to have to support a feature that might go away for other reasons.

And while we're at it, I'd like to add one more factor to the discussion: how are we planning on writing tests for macros and library augmentations (with appropriate caveats: assuming we support macros / library augmentations)? Are we going to want to analyze the augmented library and have diagnostics from the library augmentation(s) be reported, or will we analyzer the library augmentations separately? (My current expectation is that dart analyze would continue to operate as it does today and you'd only see diagnostics for the files that are explicitly included on the command-line, but that might also need to be different for the test runner.)

If desired, file an issue / follow up on the wrong human-readable output issue noted above.

Yes please. That's a bug no matter what else we decide. I don't know how much impact it's having on users, but we should at least record it so that it isn't forgotten.

[davidmorgan]

Thanks Brian! Some thoughts from my side.

... not sure who is the right person / people to decide on what the language tests do here?

I'm sure it's "people", not just "person". And I think I'm the right representative from the analyzer team. But I agree we need someone from the language team and maybe someone from the infrastructure team, depending on how big a change we might want there.

SGTM

The command line used by the language tests does report the error ...

The language tests are currently using the dartanalyzer tool. There is a strong desire (from the analyzer team at least) to remove that tool so that we have less code to maintain. The biggest difference I was previously aware of between that tool and the newer dart analyze tool is that dartanalyzer supports a mode in which the test runner can feed it files to analyze over stdin rather than on the command line. We know that we need to add that support to dart analyze (because dropping it would increase the time to run tests by far too much), but haven't been able to make the time to do so yet.

It's possible that we need to support reporting diagnostics from part files as part of the same support (that is, we could consider reporting diagnostics from part files only when in "test runner" mode).

That all sounds fine--as far as I can see it does not need to block extending the language tests using the current (deprecated) dartanalyzer tool?

Stepping back a bit--is there consensus that we want the language tests to cover (and so, be able to assert on) diagnostics in parts?

Decide if there is any issue with the current "dart analyze" behaviour or if it's working as intended / will not fix.

I think this needs to take into consideration that the answer might differ depending on whether we're asking about the behavior for users or the behavior for the test runner.

The behavior isn't consistent with how the tests are currently written, so we either need to change

• the behavior of dart analyze so that diagnostics from parts are reported when analyzing the library, or
• the way the tests are written so that we don't include parts in multiple libraries.

I know that using the same part file from multiple libraries is convenient because it means fewer duplicated files, making both the writing and maintaining of tests easier. But there isn't, as far as I'm aware, any valid language semantics for this kind of sharing, which means that the analysis server can't be used when editing these files. If we'd like people to be able to open the test directory (or some subset thereof) in an IDE, then I think the current test structure is going to be an issue. I don't want to have to support a feature that might go away for other reasons.

Having the tests stop using multi-owner parts, except where that is intentional to add test coverage for it, sounds reasonable to me and would certainly simplify the question. @sgrekhov what do you think please, would that be feasible/worthwhile?

And while we're at it, I'd like to add one more factor to the discussion: how are we planning on writing tests for macros and library augmentations (with appropriate caveats: assuming we support macros / library augmentations)? Are we going to want to analyze the augmented library and have diagnostics from the library augmentation(s) be reported, or will we analyzer the library augmentations separately? (My current expectation is that dart analyze would continue to operate as it does today and you'd only see diagnostics for the files that are explicitly included on the command-line, but that might also need to be different for the test runner.)

I expect we'll want the language tests to have at least the option to cover all the types of diagnostics that might fire, whenever they fire; so I think, yes, a "report all the diagnostics" mode would be good.

If desired, file an issue / follow up on the wrong human-readable output issue noted above.

Yes please. That's a bug no matter what else we decide. I don't know how much impact it's having on users, but we should at least record it so that it isn't forgotten.

Done: #54717

Thanks.