Process.start incorrect args double quote escaping

[danielchim]

dart --version
Dart SDK version: 2.18.1 (stable) (Tue Sep 13 11:42:55 2022 +0200) on "windows_x64"

This dart code should start an app (a.exe for example) with a json as a string:

final arguments = <String>['-bC5vLmcuaS5u={"email":"demo","password":"demo","toggle":"true","timestamp":"0"}',"-token=81a0cf770000000000000000","-config={BackendUrl:$backendUrl,Version:live}"];
 final process = await Process.start(executable, arguments, runInShell: true);

However, when I check the log in the a.exe, I find out that it will include escape characters automatically:

[Info   :demo.Core.PatchConstants] C:\Games\demo\a.exe
[Info   :demo.Core.PatchConstants] -bC5vLmcuaS5u={\"email\":\"demo\",\"password\":\"demo\",\"toggle\":\"true\",\"timestamp\":\"0\"}
[Info   :demo.Core.PatchConstants] -token=81a0cf770000000000000000
[Info   :demo.Core.PatchConstants] -config={BackendUrl:https://127.0.0.1:443,Version:live}

This caused the app to crash.

The expected behavior should be:

[Info   :demo.Core.PatchConstants] C:\Games\demo\a.exe
[Info   :demo.Core.PatchConstants] -bC5vLmcuaS5u={"email":"demo","password":"demo","toggle":true,"timestamp":0}
[Info   :demo.Core.PatchConstants] -token=81a0cf770000000000000000
[Info   :demo.Core.PatchConstants] -config={"BackendUrl":"https://127.0.0.1:443","Version":"live"}

Edit: With json and json.encode the same issue happened. I assume Process.run() will encode the double quotes with escape character as the same issue will not occurred with other languages when passing the same arguments.

const rawLoginInfo = {'email': 'demo', 'password': 'demo', 'toggle': 'true', 'timestamp': '0'};
var rawApiInfo = {'BackendUrl': 'https://127.0.0.1:443', 'Version': 'live'};
String loginInfo = jsonEncode(rawLoginInfo);
String api = jsonEncode(rawApiInfo);
final arguments = <String>['-bC5vLmcuaS5u=$loginInfo',"-token=81a0cf770000000000000000","-config=$api"];

[a-siva]

//cc @derekxu16

[derekxu16]

@danielchim I'm not able to reproduce this locally. Can you please answer the following questions so I can investigate further?

1. Do the arguments still contain escape characters if you change runInShell to false?
2. Did you run this from Command Prompt or PowerShell or something else?
3. Is a.exe an executable that you compiled yourself? If so, could you please explain how you compiled it?

Thanks!

[danielchim]

@derekxu16 The issue happens regardless of runInShell is true or false. For the second question, it's running from flutter. Also a.exe is not a program compiled by myself, it's a video game it has to take json as a launch parameter.

[derekxu16]

@danielchim Can you please try the two workarounds listed here and let me know if they work? Can you try them once with runInShell set to true and once with it set to false? Specifically, the workarounds are:

1. Combining the arguments with the executable string, and leaving the arguments array empty. So, Process.start('C:\Games\demo\a.exe -bC5vLmcuaS5u={"email":"demo", ...
2. Removing the equals signs in arguments and separating the arguments by spaces instead. So, final arguments = <String>['-bC5vLmcuaS5u', '{"email":"demo", ...

Since these are just workarounds, I also have a few more questions that will help me investigate why your original code doesn't work.

1. You aren't running the Flutter app using flutter run from the command line then right? Are you using a run button in an IDE? If so, which IDE are you using?
2. Which version of Windows are you using?

Thanks!

[danielchim]

@derekxu16 Here are the results:

tl;dr: Method 1 with setting runInShell to false works.

Method 1 code:

_launchApp() async {
    String executable = 'C:\\Games\\a-modded\\a.exe -bC5vLmcuaS5u={"email":"demo","password":"demo","toggle":"true","timestamp":"0"} -token=81a0cf770000000000000000 -config={"BackendUrl":"https://127.0.0.1","Version":"live"}';
    final arguments = <String>[];
    final process = await Process.start(executable, arguments, runInShell: true);
  }

Method 1, with runInShell set to true:

The log can't even reach to the launch parameter stage.

Method 1, with runInShell set to false:

It works as expected. (I stopped the recording, but the game launches as expected.)

Game log output:

[Info   :a.Core.PatchConstants] -bC5vLmcuaS5u={"email":"demo","password":"demo","toggle":"true","timestamp":"0"}
[Info   :a.Core.PatchConstants] -token=81a0cf770000000000000000
[Info   :a.Core.PatchConstants] -config={"BackendUrl":"https://127.0.0.1","Version":"live"}

There are no quote escaping! 🥳

Method 2 code:

_launchApp() async {
    String executable = 'C:\\Games\\a-modded\\a.exe';
    final arguments = <String>['-bC5vLmcuaS5u','{"email":"demo","password":"demo","toggle":"true","timestamp":"0"}', '-token','81a0cf770000000000000000','-config','{"BackendUrl":"https://127.0.0.1","Version":"live"}'];
    final process = await Process.start(executable, arguments, runInShell: true);
  }

Method 2, with runInShell set to true:

Game log output:

[Info   :a.Core.PatchConstants] C:\Games\a-modded\a.exe
[Info   :a.Core.PatchConstants] -bC5vLmcuaS5u
[Info   :a.Core.PatchConstants] {\"email\":\"demo\",\"password\":\"demo\",\"toggle\":\"true\",\"timestamp\":\"0\"}
[Info   :a.Core.PatchConstants] -token
[Info   :a.Core.PatchConstants] 81a0cf770000000000000000
[Info   :a.Core.PatchConstants] -config
[Info   :a.Core.PatchConstants] {\"BackendUrl\":\"https://127.0.0.1\",\"Version\":\"live\"}

Method 2, with runInShell set to false:

Game log output:

[Info   :a.Core.PatchConstants] C:\Games\a-modded\a.exe
[Info   :a.Core.PatchConstants] -bC5vLmcuaS5u
[Info   :a.Core.PatchConstants] {\"email\":\"demo\",\"password\":\"demo\",\"toggle\":\"true\",\"timestamp\":\"0\"}
[Info   :a.Core.PatchConstants] -token
[Info   :a.Core.PatchConstants] 81a0cf770000000000000000
[Info   :a.Core.PatchConstants] -config
[Info   :a.Core.PatchConstants] {\"BackendUrl\":\"https://127.0.0.1\",\"Version\":\"live\"}

I am using IntelliJ IDEA Ultimate 2021.3.3 with flutter + dart plugin, and I am using the run button on the IDE.

I am using Windows 11 22H1/22H2 professional workstation edition x64.

Also before the workaround proposed by derek, here's how I made it work in flutter with the help of ffi and win32 package:

    const String backendUrl = 'https://127.0.0.1';
    String executable = 'C:\\Games\\a-modded\\a.exe';
    String args = '-bC5vLmcuaS5u={"email":"demo","password":"demo","toggle":"true","timestamp":"0"} -token=81a0cf770000000000000000 -config={"BackendUrl":"$backendUrl","Version":"live"}';
    final exitCode = ShellExecute(NULL, TEXT("open"), TEXT(executable), TEXT(args), nullptr, SW_SHOWNORMAL);

Hope this helps someone who encounters a similar issue.

I also discovered an issue that has a similar discussion at #42571.

P.S Something noteworthy is when you add one single escape character in the double quotes, it won't add extra double quotes, until and add one more escape character looks like this:
'{\\"demo\\":\\"demo\\"}'

And the logs be like:

'{\\\"demo\\\":\\\"demo\\\"}'

[derekxu16]

I believe that the only guarantee we make is that a C program started with Process.start() will receive the correct arguments in its argv array. See

sdk/sdk/lib/_internal/vm/bin/process_patch.dart

Lines 349 to 350 in 3602351

	// Produce something that the C runtime on Windows will parse
	// back as this string.

To ensure this, we have to process the arguments passed into Process.start() to make them comply with these rules. Unfortunately, programs written in languages other than C may not follow the same rules, which I believe is what's happening with a.exe.

So, I'd say Process.start() is working as intended, but we can try improving _windowsArgumentEscape() in the future to work with as many programs as possible.