"getter 'id' was called on null" when calling addBreakpointWithScriptUri

[DanTup]

From here:

When I sent breakpoints with addBreakpointWithScriptUri like this:

[1:46:10 PM] [Observatory] [Info] ==> {"id":"608","method":"addBreakpointWithScriptUri","params":{"isolateId":"1","scriptUri":"file:///Users/dantup/Dev/Google/flutter/examples/flutter_gallery/lib/gallery/app.dart","line":64}}
[1:46:10 PM] [Observatory] [Info] ==> {"id":"609","method":"addBreakpointWithScriptUri","params":{"isolateId":"1","scriptUri":"file:///Users/dantup/Dev/Google/flutter/examples/flutter_gallery/lib/demo/animation/home.dart","line":495}}
[1:46:10 PM] [Observatory] [Info] ==> {"id":"610","method":"addBreakpointWithScriptUri","params":{"isolateId":"1","scriptUri":"file:///Users/dantup/Dev/Google/flutter/examples/flutter_gallery/lib/demo/animation/home.dart","line":496}}

I get this error:

[1:46:18 PM] [Observatory] [Error] NoSuchMethodError: The getter 'id' was called on null.
Receiver: null
Tried calling: id
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:51:5)
#1      ChromeProxyService.addBreakpointWithScriptUri (package:dwds/src/services/chrome_proxy_service.dart:182:51)
<asynchronous suspension>
#2      VmServerConnection._delegateRequest (package:vm_service/vm_service.dart:911:51)
<asynchronous suspension>
#3      _rootRunUnary (dart:async/zone.dart:1132:38)
#4      _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#5      _CustomZone.runUnaryGuarded (dart:async/zone.dart:931:7)
#6      _BufferingStreamSubscription._sendData (dart:asy

It may be related to the paths (we're sending paths, not package URIs), and there's still an open discussion about the format of these.

[jakemac53]

It looks like we don't support file: uris, @alan-knight can you look at this?

[jakemac53]

Fwiw @DanTup we basically only support package: uris or relative paths afaik right now - this is related to previous discussions we have had around uris.

I think we probably could support this though, but we will have to convert the absolute uri into a package: uri (or relative path).

[alan-knight]

Gary is confused how this ever worked for him if VS Code is always sending file: URLs and we don't have any support for those.

[grouma]

Gary is confused how this ever worked for him if VS Code is always sending file: URLs and we don't have any support for those.

In particular, using the custom version of the VS Code plugin and running with Flutter Web (not through the CLI, but with webdev daemon) worked. I was able to set breakpoints, step through files and jump to to the right location when paused.

[alan-knight]

So just to be clear on this, what do the file URLs look like for other circumstances. I think it boils down to three cases

a) as above, they're references into application files in the lib directory. These should be dealt with like package: URLs internally
b) application files but in the web or test directories. These should be dealt with like org-dartlang-app: URLs.
c) libraries in other packages. These should also be treated like package: URLs.

But it seems to me that if we can rely on dwds always having a current directory that is the root of the current package, then we can handle this without ever dealing with proper package resolution.
For (a) we can see that they're $PWD/lib/..., for (b) they're $PWD/notlib/...

So the question is what they look like for case (c) and if we can deal with that simply. Worst case we can go through the .packages file, but I'm not sure we have to.

[jakemac53]

How else would we deal with (c) without going through the .packages file?

Also note that $PWD/notlib/ could be another package (the example dir often contains a subpackage or multiple subpackages for instance).

[DanTup]

Gary is confused how this ever worked for him if VS Code is always sending file: URLs and we don't have any support for those.

In particular, using the custom version of the VS Code plugin and running with Flutter Web (not through the CLI, but with webdev daemon) worked. I was able to set breakpoints, step through files and jump to to the right location when paused.

I never actually had this work myself, but I now see the always-file-uris change was only merged into the previous release (even though it was done a while ago), so you may have been using a build from before then while I was always testing with latest code.

There are several reasons I had preferences for file URIs:

• We need to support them for things outside the lib/ folder anyway, so why have two different formats
• The .packages file isn't formally spec'd (at as far as I know, and publicly) and I've had bugs in this code before (for ex. around windows path formats, case sensitivity, comments, trailing slashes - the SDK has hand-maintained .packages files for example)
• Bugs (there have been bugs that only worked with package URIs as well as only working with file URIs, but the most recent one I remember only occurred with package URIs)
• Internally the debugger has to use file paths everywhere for its own work (eg. to open editors, to tell VS Code where to navigate to for exceptions, stack frames, clickable links in errors, etc.)
• It's possibly to run and debug Dart scripts without a pubspec or packages file

I don't think there's an obvious answer, but I do think any decisions made should documented explicitly somewhere (for ex. in the VM Service spec - FYI @bkonyi). A lot of VS Code's debugger decisions are based on "see what the IntelliJ source does" or "try both and see which work" and then when bugs crop up it's always difficult to know "is VS Code doing something wrong, or is this a bug in the VM?" which complicates getting things fixed.

[jakemac53]

We need to support them for things outside the lib/ folder anyway, so why have two different formats

In Dart libraries are identified by their URI. Historically at least the same file imported by absolute file: uri and package: uri would actually be two different libraries. So from that sense there is a real concrete difference here.

Recently I think this may have changed (at least for the application entrypoint file) where an absolute file: uri will actually be canonicalized back to a package: uri but I don't know the details. I just know the vm will spit out some sort of message if you do like dart lib/main.dart.

• The .packages file isn't formally spec'd (at as far as I know, and publicly) and I've had bugs in this code before (for ex. around windows path formats, case sensitivity, comments, trailing slashes - the SDK has hand-maintained .packages files for example)

This is the original DEP https://github.com/lrhn/dep-pkgspec/blob/master/DEP-pkgspec.md#proposal.

It isn't in the language spec because afaik the language itself isn't opinionated about uris at all - its completely up to the implementations how uris are resolved.

Internally the debugger has to use file paths everywhere for its own work (eg. to open editors, to tell VS Code where to navigate to for exceptions, stack frames, clickable links in errors, etc.)

It makes a lot of sense from an IDE perspective for sure - but it is trivial to map from a package: uri to a file: uri. The reverse is a lot more work (and also actually very error prone due to nested packages).

It's possibly to run and debug Dart scripts without a pubspec or packages file

Correct - and those should always be file: uris.

[jakemac53]

Also note that the .packages file format is actually changing slightly due to NNBD. I pushed back on this because its a very core piece of the SDK and imo it should not change in a non-breaking release of the language, but I seem to have lost that battle.

dart-lang/language#365

[DanTup]

Recently I think this may have changed (at least for the application entrypoint file) where an absolute file: uri will actually be canonicalized back to a package: uri

Yeah, I recall some changes around that (possibly dart-lang/sdk#33076). It was another step in the direction of "file paths working everywhere" which seemed great at the time :)

Also note that the .packages file format is actually changing slightly due to NNBD

Heh, apparently my nervousness is justified :D Some of those changes definitely sound like they'll break VS Code - I'll add a note on there - if they go ahead, I'll definitely need to update my code.

It makes a lot of sense from an IDE perspective for sure - but it is trivial to map from a package: uri to a file: uri. The reverse is a lot more work (and also actually very error prone due to nested packages).

But here that reverse work isn't going away, it's just being pushed to the IDE to do it instead (the editor is giving us file paths and you want package URIs). My concern is that complicated error-prone work seems better done closer to the source/SDK (in "official" code) since it's less likely to not be updated if things change (for example the change discussed above I was completely unaware of) and it's backwards/forwards compatibility demands are lessened (you can add SDK version constraints to packages and SDK tools are versioned with the SDK, but the IDE is expected to work with older SDKs as well as newer ones).

[jakemac53]

I confirmed that it is still the case that for imports at least absolute file uris and package uris are not the same thing, the following program:

import '<user-dir>/playground/lib/b.dart' as bFile;
import 'package:playground/b.dart' as bPkg;

main() {
  bPkg.printA(bFile.A());
}

Gives this CFE error from the dart vm and also an analyzer error.

bin/a.dart:5:21: Error: The argument type 'A/*1*/' can't be assigned to the parameter type 'A/*2*/'.
 - 'A/*1*/' is from 'lib/b.dart'.
 - 'A/*2*/' is from 'package:playground/b.dart' ('lib/b.dart').
Try changing the type of the parameter, or casting the argument to 'A/*2*/'.
  bPkg.printA(bFile.A());

Even if you add the cast you get a runtime failure, so the VM internally also treats these as different libraries:

Unhandled exception:
type 'A' is not a subtype of type 'A' in type cast where
  A is from file:///<user-dir>/playground/lib/b.dart
  A is from package:playground/b.dart

So, I don't see how supporting absolute file uris can be correct at all, and really shouldn't even be supported in the VM. There is no way to correctly identify which of these two libraries you want to operate on by absolute file uri.

[DanTup]

Ok, so this is weird. Seems like it works as long as you don't end up with both imported. So for example, if I take this code in VS Code:

import 'package:tttttttt/a.dart' as pA;
import '../lib/a.dart' as fA;

main(List<String> arguments) {
  pA.foo(new pA.Bar());
  fA.foo(new fA.Bar());
}

And put a breakpoint inside foo in a.dart and run it, the VM rejects the breakpoint and it's never hit:

{
	"jsonrpc": "2.0",
	"error": {
		"code": 102,
		"message": "Cannot add breakpoint",
		"data": {
			"request": {
				"method": "addBreakpointWithScriptUri",
				"params": {
					"isolateId": "isolates\/1370140353987883",
					"scriptUri": "file:\/\/\/Users\/dantup\/Desktop\/tttttttt\/lib\/a.dart",
					"line": "4"
				}
			},
			"details": "addBreakpointWithScriptUri: Cannot add breakpoint at line '4'"
		}
	},
	"id": "10"
}

However, if I comment out either of those imports and the corresponding line, the breakpoint is hit.

So it seems like the VM may be happy to accept either, as long as they can only resolve to one library. If there are multiple, it rejects the breakpoint (I don't know if there are other places it'd do the same).

[jakemac53]

Correct, this is exactly why the vm shouldn't be supporting absolute file uris for things that are actually loaded with a package: uri. It is actually lying about the uri for the library it returns to you. As soon as both exist in the program it now can't disambiguate.

[alan-knight]

So on the web, does that mean that a package: URL and an http: URL would be two different things? A file: URL isn't really possible there, so we'd either have to reject it outright or turn it into one of the others.

[DanTup]

It is actually lying about the uri for the library it returns to you. As soon as both exist in the program it now can't disambiguate.

I think it's more complicated than this. It is possible for the user to load a file using a file URI (as the above shows), so blindly converting them to package: URIs also seems fraught with danger. What if the loaded one is a file:// URI and in VS Code we convert it to package: URI? No breakpoints will work :-/

[jakemac53]

Try and set it both ways? lol.

[jakemac53]

On a serious not that is a good point though, it is unreasonable for the IDE to know all the transitive ways a library was imported in an app.

In general, by convention at least, nothing under lib of a package should be resolved to a file: uri, they should always be package: uris to avoid the A is not an A issue.

The examples I brought up really do indicate a user error imo, and would not even be allowed in a web context anyways.

That being said most defensive thing the IDE could do is try it both ways. In theory that would be the correct thing to do in the manufactured case above to set the breakpoint in each version of the library as well.

That obviously kind of sucks though.

[jakemac53]

I think in general given all the complexities here I think I have come around to just adding in support for file: uris.

When we are constructing the LibraryRef instances we can create a reverse mapping of absolute file uri to library and do O(1) lookups. This will be significantly less error prone than other strategies because at that time we know exactly what the absolute file uri is for a package: uri.

This also brings our behavior in line with the VM, for better or worse, which has generally been our overarching strategy.

It isn't technically correct, but it makes things easier on the ide's and should work out of the box for more tools that work today.

[DanTup]

Try and set it both ways? lol.

That's actually what we used to do - but there were other complexities (because we have to map breakpoints between VS Code and the VM, and if this mapping isn't one-to-one, you get interesting behaviour like having to delete a breakpoint twice for it to disappear... it's stuff we can work around, but the easier fix was to just not do it :-))

That being said most defensive thing the IDE could do is try it both ways. In theory that would be the correct thing to do in the manufactured case above to set the breakpoint in each version of the library as well.

That's true, but you could also argue the same for the VM. If the IDE is just going to send both, the VM could just interpret one of them as both. It seems weird to teach the IDE that "we're just going to always treat these things as the same" if the VM could do it and shield the IDE from the complexity.

In general, by convention at least, nothing under lib of a package should be resolved to a file: uri,

Yeah, it'd be nice if this was enforced and there was never any ambiguity.. I think that ship has sailed though. That said, if we had a lint that enforced it or something, it would give more weight for IDEs only working with one way.

I think in general given all the complexities here I think I have come around to just adding in support for file: uris.

Sounds good to me 😄

If it doesn't work out, I'm not against changing things - as long as we can write them down explicitly. There's a lot of ambiguity in these things (because there aren't really many clients) and it makes it hard to ensure they work the same (I've spent a lot of time chasing bugs around breakpoints not being hit and the like). Of course, I'd prefer to keep that complexity closer to the source though :-)

[alan-knight]

Interesting side point. In the VM it appears these are actually two separate libraries. If I make a program that imports both package: and file: and debug in observatory they are two separate libraries with two separate URIs. Observatory won't let me set a breakpoint on either one.

[DanTup]

Observatory won't let me set a breakpoint on either one.

That matches what I found from VS Code above. I don't know it if's always been that way, or has happened as fixes have been made to handle paths/package URIs better. It doesn't feel quite right, but then I'm not sure that loading the same file under both URIs is right either. I can't imagine that situation is ever done deliberately (besides us testing!), if it happens it's likely a mistake on the users part (and often causes weird issues - like ending up library-level state duplicated).

[alan-knight]

@DanTup
I've landed file URL support in #676
Hopefully this works for you.

[DanTup]

Great, thanks! Do I need to wait for a release to try this out? I tried running flutter update-packages --force-upgrade but I still see the same error (I don't think the latest version on pub has this change).

[alan-knight]

Unfortunately it's a bit awkward right now. There's another change that's breaking for Flutter, so we need to fix that. I'll see about getting that done.

[alan-knight]

I've created a Flutter pull request. flutter/flutter#41805 that you'll need in order to test this.

[alan-knight]

Closing this as it's published. If there are still problems re-open or file another bug.

[DanTup]

Awesome, thanks! I'll try it out once that PR lands.