Uri.toString() for "mailto" does not encode spaces properly (`+` instead of `%20`)

[Delgan]

Hi.

This ticket was first opened on flutter bug tracker: flutter/flutter#68406

It seems Uri.toString() convert spaces to + in subject and body for mailto scheme.

final uri = Uri(
  scheme: 'mailto',
  path: 'smith@example.com',
  queryParameters: {
    'subject': 'Example Subject & Symbols are allowed!'
  }
);

print(uri.toString());
// mailto:smith@example.com?subject=Example+Subject+%26+Symbols+are+allowed%21

This seems to be a problem, when the link is opened with url_launcher for example then the subject displayed in the mail app is like this:

Example+Subject+&+Symbols+are+allowed!

For some reason, the app does not handle + symbols in the uri as expected. However, it works fine if %20 is used instead.

I tested this with url_launcher v5.7.5, flutter v1.23.0, dart v2.10.0, Android v8.0.0 and opening the link with Gmail app.

Should the spaces be automatically converted to %20 instead to ensure portability?

[lrhn]

The Dart Uri class does not recognize most schemes specially. The mailto: scheme is treated the same as any other scheme, so the query part is encoded using URL encoding, which is why it uses + for spaces.

[SamadiPour]

Same here. How to fix it?

[lrhn]

The lazy approach would be:

print(uri.toString().replaceAll("+", "%20"));

A more intricate approach could be:

final uri = Uri.parse('mailto:smith@example.com?subject=' + 
   'Example Subject & Symbols are allowed!'.replaceAll(" ", "%20"));

print(uri.toString());

or

final uri = Uri(
  scheme: 'mailto',
  path: 'smith@example.com',
  query: 
    'subject=Example Subject & Symbols are allowed!'.replaceAll(" ", "%20")
);

print(uri.toString());

There currently is no way to use the queryParameters argument and get what you want. There is no way to configure the URI to not use standard URL encoding for query parameters.

(If we do a complete rewrite of the Uri class at some point, which we want to for several reasons, mainly compatibility with the browser Url class, then we will likely start recognizing "web" URIs - http and web sockets - specifically, and not treat all other URIs as if they are necessarily the same as HTTP. Until such time, we are unlikely to make any breaking changes to the Uri class.)

[Delgan]

I'm personally constructing the uri string myself by calling Uri.encodeComponent() for each parameter. It encodes " " to "%20" while Uri.encodeQueryComponent() (which I guess is used internally by Uri.toString()) encodes it to "+".

final subject = Uri.encodeComponent("Example Subject & Symbols are allowed!")
final uri = "mailto:smith@example.com?subject=$subject";

It seems to work fine.

[stuartmorgan-g]

The Dart Uri class does not recognize most schemes specially. The mailto: scheme is treated the same as any other scheme, so the query part is encoded using URL encoding, which is why it uses + for spaces.

This is backwards though, as the generic form of URL encoding is percent encoding (click on your link there, then click on "URL encoding" in that text, which leads to https://en.wikipedia.org/wiki/Percent-encoding). The issue isn't that mailto isn't getting special handling, it's that all schemes are being treated as if they are HTML form query strings, which is where the use of + comes from.

There is no way to configure the URI to not use standard URL encoding for query parameters.

The correct statement is that there is no way to configure Uri to use standard URL encoding for query parameters, rather than applying HTML form encoding to every scheme.

A more intricate approach could be:

final uri = Uri.parse('mailto:smith@example.com?subject=' + 
   'Example Subject & Symbols are allowed!'.replaceAll(" ", "%20"));

print(uri.toString());

or

final uri = Uri(
  scheme: 'mailto',
  path: 'smith@example.com',
  query: 
    'subject=Example Subject & Symbols are allowed!'.replaceAll(" ", "%20")
);

print(uri.toString());

These suggestions are both incorrect, as they will not encode anything else. Notably in this case, the subject will just be "Example Subject ", because the unencoded & will end that query argument.

Until such time, we are unlikely to make any breaking changes to the Uri class.

Perhaps an alternate constructor that does the right thing as an interim solution? This issue makes it very hard for people to correctly use the Uri class for anything that's not an http* URI. (Exhibit A: the fact that your example workarounds are not actually right, and will break things in other ways.)

Or, alternately, switch on the scheme and keep the existing behavior for http/https, and do a breaking change for any others? While it is a breaking change, it's hard to imagine many scenarios where anyone would prefer the current behavior for other schemes.

[stuartmorgan-g]

(Also, perhaps the encodeComponent docs could be updated in the short term; currently they say "For encoding the query part consider using encodeQueryComponent." which is actively misleading advice for any non-http URI. It could say "For encoding the query part of an HTTP or HTTPS URI, consider ...")

[stuartmorgan-g]

The simplest safe solution I can see at this point to use Uri for anything that has query parameters but isn't an http* URL is to manually re-implement what queryParameters should do inline, using encodedComponent. E.g.:

final parameters = <String, String>{
  'subject': 'Example Subject & Symbols are allowed!',
  'body': 'Here too! &+=',
};
final uri = Uri(
  scheme: 'mailto',
  path: 'smith@example.com',
  query: parameters.entries.map((e) => '${Uri.encodeComponent(e.key)}=${Uri.encodeComponent(e.value)}').join('&'),
);
print(uri.toString());

For url_launcher I think we'll probably need to provide a utility method to do that query encoding, since that's a lot to put in a recommended code samples.

[Hixie]

https://url.spec.whatwg.org/ is the controlling standard here and it describes our current behaviour as invalid. Since we are not following the standard, we should fix our behaviour accordingly.

[lrhn]

Changing our behavior to match the whatwg spec was one of our plans for Dart 2.0, but it didn't pan out (we significantly over-estimated how much change we could do in the time available). See #29420

It's still a worthy goal, and would make web compilers save a lot of code, but also a highly breaking change to anyone using the existing Uri class.
One strategy (which we have used before with varying success) could be to copy the existing Uri class to a package, and re-implement the API with whatwg-URL compatible semantics in the platform libraries. People depending heavily on the existing behavior can migrate to the package then.

It's going to be a big undertaking. The Uri class is the largest class in the platform libraries, by a significant margin. The uri.dart file is larger than all other .dart files in the SDK except the auto-generated html_dart2js.dart. It's bigger than http_impl.dart!

(Reducing the complexity was one of the reasons for rewriting Uri, to not have to do copious amounts of up-front normalization by default).

[stuartmorgan-g]

Can you comment on the possiblity of doing one of the two small changes I suggested above (a new named constructor for non-web Uris, or a breaking change for the constructor just for non-web schemes)?

I understand there are other issues that make a major rewrite desirable, but if we could avoid letting the perfect being the enemy of the good in fixing this particular issue in the short term it would make it feasible to use Uri directly for common cases like mailto:, which currently is not the case.

[Hixie]

a highly breaking change to anyone using the existing Uri class.

I don't see how making our behaviour less broken could be considered a breaking change. Could you elaborate on the flow that you're concerned would break?