[Bug]: Locator `.check()` behavior changed with upgrade from 1.50 to 1.54

[paunovic]

Version

1.54

Steps to reproduce

On latest (1.54) version, sometimes, page.locator().check() throws following error, we have few tests where this started consistently happening:

playwright._impl._errors.Error: Locator.check: Clicking the checkbox did not change its state
Call log:
  - waiting for locator(".shared-checkbox.item .input").nth(1)
    - locator resolved to 
  - attempting click action
    - waiting for element to be visible, enabled and stable
    - element is visible, enabled and stable
    - scrolling into view if needed
    - done scrolling
    - performing click action
    - click action done
    - waiting for scheduled navigations to finish
    - navigations have finished

Tests open a webpage, and click few checkboxes with page.locator().check(), e.g.:

def test_webpage(page, test_eligibility_session):

    p = pom.Homepage(page, plan_id=plan.id)

    logger.info("check that associated group is linked")
    with p.page.expect_response(lambda resp: resp.request.method == "POST" and resp.url.endswith("/group/list")):
        p.page.locator(p.ASSOCIATED_GROUP).click()

    expect(p.table.rows[0]["Group Name"]).to_have_text(group_version.name)

    logger.info("fill Effective date")
    p.page.locator(p.EFFECTIVE_DATE_INPUT).fill(arrow.now().shift(days=-1).strftime("%m/%d/%Y"))

    logger.info("fill Termination date")
    p.page.locator(p.TERMINATION_DATE_INPUT).fill(arrow.now().shift(years=10).strftime("%m/%d/%Y"))

    for election in p.page.locator(".shared-checkbox.item .input").all():
        election.check()

Expected behavior

Checkbox should be checked without errors.

Actual behavior

Checkbox state gets updated just fine, when watching a recording, I see that checkbox is actually being checked, but subsequent checkbox action verifications fail with above error.

Additional context

We have been using 1.50 successfully with over >900 tests, but with recent attempts to upgrade to 1.54 version and with no other changes to our framework we are consistently getting this error across only a few tests.

I'm wondering if it's something with our UI but given that 1.50 and all previous versions worked fine, suspecting it's something with recent 1.54 version that have changed this behavior.

There is also similar issue here: #13470

Environment

- Operating System: Ubuntu 22.04
- CPU: arm64
- Browser: chromium
- Python Version: 3.12.3

[Skn0tt]

Hmm, from first sight this sounds like Playwright caught a bug in your application. If you believe otherwise, could you provide us with a reproduction page and test?

[paunovic]

Agreed to an extent, it always fails on a few specific tests and UI pages, but not always on the same checkbox, so still seems slightly random. I suspect there's an element overlaying checkboxes but haven't been able to confirm this. So unsure if I can make a reproducible example.

However this also specifically started happening after we tried to upgrade from 1.50 to 1.54/1.55, and I've seen similar (old) bug reports floating around without solutions, so just trying to pinpoint the issue here. On 1.50 everything works fine.

I'll try to make a repro these days if possible, would also appreciate insight if possible if anything changed between these two versions when it comes to checkbox behavior.

Thanks!

[Skn0tt]

I'm not aware of any specific changes to the behaviour in these versions, but if it's from overlay elements and it's flaky, then there's a good chance it's just from slight browser behaviour changes.