RemoteWebDriver cannot be cast to HasDevTools

[boris779]

🐛 Bug Report

It's not possible to use the DevTools with the RemoteDriver. I was told to use the HasDevTools Interface but it still not working when using a RemoteWebDriver (with a Grid)

java.lang.ClassCastException: class org.openqa.selenium.remote.RemoteWebDriver cannot be cast to class org.openqa.selenium.devtools.HasDevTools (org.openqa.selenium.remote.RemoteWebDriver and org.openqa.selenium.devtools.HasDevTools are in unnamed module of loader 'app')

Detailed steps to reproduce the behavior:

webDriver is RemoteWebDriver which is used in a SeleniumGrid Environment.

Java:
DevTools devTools = ((HasDevTools) webDriver).getDevTools();

Kotlin:
val devTools: DevTools = (webDriver as HasDevTools).devTools

Used Selenium Version: 4.0.0-rc-1 (also tested with beta-3 and beta-4)

Environment

OS: Ubuntu 21
Browser: Chrome
Browser version: > 87
Browser Driver version: >87
Language Bindings version: Java

[pujagani]

Thank you for providing the details. I am sharing a working code snippet below that uses Augmenter. Essentially, Augmenter enhances the RemoteWebDriver to allow the use for DevTools apis.

public class ChromeDevToolsAugmenter {

  public static void main(String[] args) throws MalformedURLException {

    ImmutableCapabilities capabilities = new ImmutableCapabilities("browserName", "chrome");
    WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444"), capabilities);

    driver = new Augmenter().augment(driver);

    driver.get("https://www.duckduckgo.com");
    DevTools devTools = ((HasDevTools) driver).getDevTools();
    devTools.createSession();

    devTools.send(Log.enable());
    devTools.send(Browser.setWindowBounds(new WindowID(1), new Bounds(
      Optional.of(20),
      Optional.of(20),
      Optional.of(20),
      Optional.of(20),
      Optional.of(WindowState.NORMAL))));

    driver.quit();
  }
}

Please try it out with Augmenter and provide feedback if the error still persists. Will be happy to dig into it.

[boris779]

Hi!.
No sorry, I tried some code examples but still get the same error.

java.lang.ClassCastException: class org.openqa.selenium.remote.RemoteWebDriver cannot be cast to class org.openqa.selenium.devtools.HasDevTools (org.openqa.selenium.remote.RemoteWebDriver and org.openqa.selenium.devtools.HasDevTools are in unnamed module of loader 'app')

[boris779]

I added my source code to GitHub.
https://github.com/boris779/selenium4_caps_bug/tree/devTools

Beware that you switch to the devTools Branch, I created the repo for another bug

[boris779]

There is also a runconfig in the ressources folder called "Dev Tools Selenium 4 remote chrome" which maybe help you

[pujagani]

Thank you for sharing the details and the code. Appreciate it. I will try it out on my machine and revert. However, based on the error message shared, I had a question. What Java version do you use? I am suspecting a Java version issue, since Java 9 introduced modules. Similarly, Java 16 has modules as well that causes a similar error as you shared.

[pujagani]

I have run the Cucumber run-config as mentioned and I am unable to reproduce the issue. All tests pass as expected. The only thing I changed in the code is the platform since I am on MacOS. The tests are passing after multiple runs.

Grid is creating a CDP session and upgrading the connection to use WebSocket correctly.

OS: MacOS
Browser: Chrome
Browser version: 93
Browser Driver version: 93
Language Bindings version: rc-1, beta-3
Java version: 11

I see that the cucumber-java8 used in the project does not support Java 12+, so that eliminates Java 16 module issue. Is there anything else I should be considering? or anything else I can do to recreate the issue?

[boris779]

Hi!

I tried it with JDK 17 and JDK 11. Both not working, but get different error messages/exceptions.

To get cucumber-java8 running with Java12+ you have to change the "Before lunch" actions in the run config. It's important to:

• clean when changing the JDK Version
• add a maven Goal: install

This is because of https://stackoverflow.com/questions/67787645/cucumber-kotlin-illegalstateexception-when-launching-test-via-intellij/67817085#67817085

With JDK-17:
Step failed org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://172.25.0.1:4444/session/3816a89326169d124b0b109dc4ad22fe/se/cdp Build info: version: '4.0.0-rc-1', revision: 'bc5511cbda' System info: host: 'enterprise', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.11.0-36-generic', java.version: '17-ea' Driver info: driver.version: unknown at org.openqa.selenium.remote.http.netty.NettyWebSocket.<init>(NettyWebSocket.java:104) at org.openqa.selenium.remote.http.netty.NettyWebSocket.lambda$create$3(NettyWebSocket.java:137) at org.openqa.selenium.remote.http.netty.NettyClient.openSocket(NettyClient.java:127) at org.openqa.selenium.devtools.Connection.<init>(Connection.java:73) at org.openqa.selenium.devtools.SeleniumCdpConnection.<init>(SeleniumCdpConnection.java:34) at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$0(SeleniumCdpConnection.java:56) at java.base/java.util.Optional.map(Optional.java:260) at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:54) at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:47) at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:48) at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29) at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:186) at at.co.boris.webdriversession.webdriverfactory.RemoteChromeDriverFactory.createDriver(RemoteChromeDriverFactory.java:45) at at.co.boris.webdriversession.webdriverfactory.DriverFactory.createWebDriver(DriverFactory.java:43) at at.co.boris.webdriversession.WebDriverSession.<init>(WebDriverSession.java:22) at at.co.boris.webdriversession.WebDriverSessionStore.getOrCreate(WebDriverSessionStore.java:18) at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriverSession(AbstractStepDefs.java:62) at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriver(AbstractStepDefs.java:56) at at.co.boris.stepdefinitions.WikipediaSteps.lambda$new$0(WikipediaSteps.java:26) at ✽.the start page is loaded(file:///home/boris/repos/selenium4_caps_bug/src/test/resources/features/wikipedia.feature:6)

[boris779]

With JDK-11 I get the same error:

`/usr/lib/jvm/java-11-openjdk-amd64/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:46279,suspend=y,server=n -Dbrowser=remote_chrome -DbaseURL=http://www.wikipedia.at -Ddriver.version=93.0 -Dbrowser.version=93.0 -Dselenium.grid=http://localhost:4444 -Dorg.jetbrains.run.directory=/home/boris/repos/selenium4_caps_bug/src/test/resources/features -javaagent:/home/boris/idea/plugins/java/lib/rt/debugger-agent.jar -Dfile.encoding=UTF-8 -classpath /home/boris/repos/selenium4_caps_bug/target/test-classes:/home/boris/.m2/repository/io/cucumber/cucumber-java8/6.10.4/cucumber-java8-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/cucumber-core/6.10.4/cucumber-core-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/cucumber-gherkin/6.10.4/cucumber-gherkin-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/cucumber-gherkin-messages/6.10.4/cucumber-gherkin-messages-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/messages/15.0.0/messages-15.0.0.jar:/home/boris/.m2/repository/io/cucumber/tag-expressions/3.0.1/tag-expressions-3.0.1.jar:/home/boris/.m2/repository/io/cucumber/cucumber-expressions/10.3.0/cucumber-expressions-10.3.0.jar:/home/boris/.m2/repository/io/cucumber/datatable/3.5.0/datatable-3.5.0.jar:/home/boris/.m2/repository/io/cucumber/cucumber-plugin/6.10.4/cucumber-plugin-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/docstring/6.10.4/docstring-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/html-formatter/13.0.0/html-formatter-13.0.0.jar:/home/boris/.m2/repository/io/cucumber/create-meta/4.0.0/create-meta-
.
.
.

Testing started at 16:23 ...
Connected to the target VM, address: '127.0.0.1:46279', transport: 'socket'
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
2021-09-27 | 16:23:54.309 | main | INFO | a.c.b.s.WikipediaSteps | ID: | Constructor AbstractStepDefs
2021-09-27 | 16:23:54.462 | main | INFO | a.c.b.s.WikipediaSteps | ID: | Before All Once
Sep 27, 2021 4:23:56 PM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer
INFO: Using OpenTelemetry for tracing
Sep 27, 2021 4:24:11 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Sep 27, 2021 4:24:15 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
INFO: Found exact CDP implementation for version 93
Sep 27, 2021 4:24:15 PM org.openqa.selenium.remote.http.WebSocket$Listener onError
WARNING: Invalid Status code=400 text=Bad Request
java.io.IOException: Invalid Status code=400 text=Bad Request
at org.asynchttpclient.netty.handler.WebSocketHandler.abort(WebSocketHandler.java:92)
at org.asynchttpclient.netty.handler.WebSocketHandler.handleRead(WebSocketHandler.java:118)
at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)

Step failed
org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://172.25.0.1:4444/session/49f231a2e40bb712aab1daddae4df952/se/cdp
Build info: version: '4.0.0-rc-1', revision: 'bc5511cbda'
System info: host: 'enterprise', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.11.0-36-generic', java.version: '11.0.11'
Driver info: driver.version: unknown
at org.openqa.selenium.remote.http.netty.NettyWebSocket.(NettyWebSocket.java:104)
at org.openqa.selenium.remote.http.netty.NettyWebSocket.lambda$create$3(NettyWebSocket.java:137)
at org.openqa.selenium.remote.http.netty.NettyClient.openSocket(NettyClient.java:127)
at org.openqa.selenium.devtools.Connection.(Connection.java:73)
at org.openqa.selenium.devtools.SeleniumCdpConnection.(SeleniumCdpConnection.java:34)
at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$0(SeleniumCdpConnection.java:56)
at java.base/java.util.Optional.map(Optional.java:265)
at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:54)
at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:47)
at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:48)
at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29)
at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:186)
at at.co.boris.webdriversession.webdriverfactory.RemoteChromeDriverFactory.createDriver(RemoteChromeDriverFactory.java:45)
at at.co.boris.webdriversession.webdriverfactory.DriverFactory.createWebDriver(DriverFactory.java:43)
at at.co.boris.webdriversession.WebDriverSession.(WebDriverSession.java:22)
at at.co.boris.webdriversession.WebDriverSessionStore.getOrCreate(WebDriverSessionStore.java:18)
at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriverSession(AbstractStepDefs.java:62)
at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriver(AbstractStepDefs.java:56)
at at.co.boris.stepdefinitions.WikipediaSteps.lambda$new$0(WikipediaSteps.java:26)
at ✽.the start page is loaded(file:///home/boris/repos/selenium4_caps_bug/src/test/resources/features/wikipedia.feature:6)`

[boris779]

To simplify I also tried your lightweight example and created a new repository

https://github.com/boris779/selenium4_devtools

[pujagani]

To simplify I also tried your lightweight example and created a new repository

https://github.com/boris779/selenium4_devtools

Thank you for providing the details. However, still unable to reproduce the issue with the new repository as well. Which version of Grid are you running? I am running tests using the Java Selenium binding rc-1. The Grid runs in the Standalone mode: java -jar selenium-server-4.0.0-rc-1.jar standalone.

package at.co.boris.devtools;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v92.browser.Browser;
import org.openqa.selenium.devtools.v92.browser.model.Bounds;
import org.openqa.selenium.devtools.v92.browser.model.WindowID;
import org.openqa.selenium.devtools.v92.browser.model.WindowState;
import org.openqa.selenium.devtools.v92.log.Log;
import org.openqa.selenium.remote.Augmenter;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.MalformedURLException;
import java.net.URL;
import java.util.Optional;

import static org.openqa.selenium.remote.CapabilityType.BROWSER_VERSION;

public class Main {

    public static void main(String[] args) throws MalformedURLException {

        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability("browserName", "chrome");
      //  capabilities.setCapability(BROWSER_VERSION, "94.0");

        WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4441"), capabilities);

        driver = new Augmenter().augment(driver);

        driver.get("https://www.duckduckgo.com");
        DevTools devTools = ((HasDevTools) driver).getDevTools();
        devTools.createSession();

        devTools.send(Log.enable());
        devTools.send(Browser.setWindowBounds(new WindowID(1), new Bounds(
                Optional.of(20),
                Optional.of(20),
                Optional.of(20),
                Optional.of(20),
                Optional.of(WindowState.NORMAL))));

        driver.quit();


    }
}

I do not explicitly set the version on the client and the server-side and let the Grid figure out the best match.
OS: MacOS
Browser: Chrome
Browser version: 94.0.4606.61
Browser Driver version: 94.0.4606.61
Language Bindings version: rc-1
Java version: 9, 11

[boris779]

That's really weird, I was also running the selenium grid rc1 but the docker version.

java -jar selenium-server-4.0.0-rc-1.jar standalone -D selenium/standalone-chrome:93.0 '{"browserName": "chrome", "browserVersion": "94.0", "platformName": "linux"}' -D selenium/standalone-firefox:latest '{"browserName": "firefox"}' --detect-drivers false

I will keep on searching

[boris779]

Now we found the issue.

It works with the standalone version of selenium grid rc-1 but not with the docker-version.