Support for headless Target API

[jameshartig]

Ideally we could use the Target API. Apparently that means interacting with /json/browser endpoint (and essentially a totally new API), which means it isn't really feasible for me this library as-is, right?

[cyrus-and]

I'm afraid I'm not fully understanding what you say. Do you suggest using the Target API instead of the HTTP endpoint for CDP.New(), CDP.Close(), etc. ?

The problem with the Target API is that you already need a target in order to use it, while this may make sense for the --headless case it doesn't for a regular Chrome session.

Said that, nothing prevents you to use the Target API directly, i.e., you perform the connection just once with CDP(...) then use the API to manage your targets.

---

What is the /json/browser endpoint anyway? According to devtools_http_handler.cc there is no such thing.

[jameshartig]

What is the /json/browser endpoint anyway? According to devtools_http_handler.cc there is no such thing.

Woops I meant /devtools/browser.

Anyways, I managed to figure it out with help from:
https://groups.google.com/a/chromium.org/forum/?utm_medium=email&utm_source=footer#!msg/headless-dev/2tWMx4k-5ig/ymhFPDeMDAAJ

I didn't realize that once you called Target.createTarget you could use List to get the websocket URL for a target that was created.

[cyrus-and]

The problem with the Target API is that you already need a target in order to use it, while this may make sense for the --headless case it doesn't for a regular Chrome session.

OK, this is not accurate, I didn't know about the /devtools/browser WebSocket endpoint... The special DevTools target (ws://localhost:9222/devtools/browser) can be used to manage the targets even when there are no targets (tabs) open.

And as you saw in that Google Groups discussion, it can be used as any other target with chrome-remote-interface. For example:

async function newTarget(url) {
    const client = await CDP({tab: 'ws://localhost:9222/devtools/browser'});
    const info = await client.Target.createTarget({url});
    await client.close();
    return info.targetId;
}

newTarget('http://reddit.com')
    .then(console.log)
    .catch(console.error);

Some considerations:

• /devtools/browser only works properly in headless mode, e.g., 'Target.createTarget' wasn't found during a regular Chrome session;

• anyway Target.getTargets never works via /devtools/browser, headless or not (same reason as above);

• hence, I'm reluctant to incorporate the Target API (via /devtools/browser) directly in CRI yet because it's still in an experimental stage.

Related: #83

[jameshartig]

anyway Target.getTargets never works via /devtools/browser, headless or not (same reason as above);

Filed: https://bugs.chromium.org/p/chromium/issues/detail?id=697048

hence, I'm reluctant to incorporate the Target API (via /devtools/browser) directly in CRI yet because it's still in an experimental stage.

I agree and didn't realize initially how experimental it was. By using CDP({tab: 'ws://localhost:9222/devtools/browser'}) I've managed to get everything I need working.

Once the experimental-ness is gone, I could see a "browser" object that exposes this functionality but the headless restriction is going to be hard to inform the user about.

[cyrus-and]

Filed: https://bugs.chromium.org/p/chromium/issues/detail?id=697048

Thanks!

Once the experimental-ness is gone, I could see a "browser" object that exposes this functionality but the headless restriction is going to be hard to inform the user about.

Ideally, it should be part of the current API (CDP.New(), etc.) so to be completely transparent to the users, it would use both the HTTP endpoint and the WebSocket (/devtools/browser) Target API depending on the case.

[cyrus-and]

For completeness here is the reason why CDP.Protocol() doesn't work in headless mode: https://bugs.chromium.org/p/chromium/issues/detail?id=696457 (fixed now).

[incubus]

Hi!

Does anyone know how to useTarget.getTargets method? I get

(node:48883) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: 'Target.getTargets' wasn't found for this code:

    const browser = await CDP({target: 'ws://127.0.0.1:9222/devtools/browser'});
    const {Target} = browser;
    const targets = await Target.getTargets();
    console.log(targets);

Everything is ok with Target.createBrowserContext and Target.createTarget.

    const browser = await CDP({target: 'ws://127.0.0.1:9222/devtools/browser'});
    const {Target} = browser;
    const {browserContextId} = await Target.createBrowserContext();
    console.log(browserContextId);
    const target = await Target.createTarget({url: url, browserContextId });

Thanks!

[incubus]

Sorry, found this https://bugs.chromium.org/p/chromium/issues/detail?id=697048.

Instead of ws://localhost:9222/devtools/browser I needed to follow instructions here: cyrus-and/chrome-har-capturer@53bd59c also reference: #288