Skip to content

iOS screenshot error in JIMP when cropping an image received from Appium #783

Open
@AfLosada

Description

@AfLosada

Environment (please complete the following information):

  • Node.js version: 20.10.0
  • NPM version: 10.2.3
  • Browser name and version: Safari
  • Platform name and version: iOS phone
  • WebdriverIO version: 9.12.0
  • @wdio/visual-service version: 6.3.3

Config of WebdriverIO + @wdio/visual-service

    [
      'visual',
      {
        // Service options
        autoSaveBaseline: true,
        createJsonReportFiles: true,
      } satisfies VisualServiceOptions, // Ensures type safety
    ]

Describe the bug

Taking a screenshot of a real iPhone 12 fails when Jimp tries to crop the image.

The take screenshot command is sent succesfully and is received by webdriverIO:

Image

but when it tries to call the image-comparison logic Jimp breaks:

Image

To Reproduce
Steps to reproduce the behavior:

The case that I'm testing is similar to the following:

  1. Setup a Selenium grid with an appium node that is connected to an iPhone
  2. Capability should be 'safari'
  3. Configure webdriverio to have a test that runs this:
  await expect(browser).toMatchScreenSnapshot(
    `video-playing-${browserName}`,
    0,
    { scaleImagesToSameSize: false }
  )

Expected behavior

  1. The image is able to be saved and processed by webdriverIO.

Log

[0-0] 2025-03-18T19:52:22.944Z INFO webdriver: [POST] http://localhost:4444/wd/hub/session/bff1e65d-9f59-4abb-89d4-23e1d279301d/execute/sync
[0-0] 2025-03-18T19:52:22.944Z INFO webdriver: DATA {
[0-0]   script: 'setCustomCss(...) [1975 bytes]',
[0-0]   args: [
[0-0]     {
[0-0]       addressBarPadding: 0,
[0-0]       disableBlinkingCursor: false,
[0-0]       disableCSSAnimation: false,
[0-0]       id: 'pic-css',
[0-0]       toolBarPadding: 0
[0-0]     }
[0-0]   ]
[0-0] }
[0-0] 2025-03-18T19:52:23.081Z INFO webdriver: RESULT null
[0-0] 2025-03-18T19:52:23.583Z INFO webdriver: COMMAND executeScript(<fn>, <object>)
[0-0] 2025-03-18T19:52:23.584Z INFO webdriver: [POST] http://localhost:4444/wd/hub/session/bff1e65d-9f59-4abb-89d4-23e1d279301d/execute/sync
[0-0] 2025-03-18T19:52:23.584Z INFO webdriver: DATA { script: 'getScreenDimensions(...) [1668 bytes]', args: [] }
[0-0] 2025-03-18T19:52:23.664Z INFO webdriver: RESULT {
[0-0]   dimensions: {
[0-0]     body: { scrollHeight: 260, offsetHeight: 260 },
[0-0]     html: {
[0-0]       clientHeight: 260,
[0-0]       clientWidth: 375,
[0-0]       scrollHeight: 260,
[0-0]       scrollWidth: 375,
[0-0]       offsetHeight: 260
[0-0]     },
[0-0]     window: {
[0-0]       innerWidth: 375,
[0-0]       innerHeight: 260,
[0-0]       isLandscape: true,
[0-0]       outerHeight: 260,
[0-0]       outerWidth: 375,
[0-0]       devicePixelRatio: 3,
[0-0]       screenWidth: 375,
[0-0]       screenHeight: 812
[0-0]     }
[0-0]   }
[0-0] }
[0-0] 2025-03-18T19:52:23.665Z INFO webdriver: COMMAND takeScreenshot()
[0-0] 2025-03-18T19:52:23.665Z INFO webdriver: [GET] http://localhost:4444/wd/hub/session/bff1e65d-9f59-4abb-89d4-23e1d279301d/screenshot
[0-0] 2025-03-18T19:52:24.037Z INFO webdriver: RESULT iVBORw0KGgoAAAANSUhEUgAABGQAAAmECAIAAADMhS/QAAAAAXNSR0IArs4c6...
[0-0] 2025-03-18T19:52:24.458Z WARN @wdio/visual-service:webdriver-image-comparison:images: 
[0-0] #####################################################################################
[0-0]  THE RESIZE DIMENSION RIGHT=0 MADE THE CROPPING GO OUT OF THE SCREEN SIZE
[0-0]  RESULTING IN A RIGHT CROP POSITION=2436.
[0-0]  THIS HAS BEEN DEFAULTED TO '1124'
[0-0] #####################################################################################
[0-0] 
[0-0] RangeError in "DRM Tests.should validate DRM in iOS Safari"
RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 10952252. Received 10952256
    at boundsError (node:internal/buffer:88:9)
    at Buffer.readUInt32BE (node:internal/buffer:311:5)
    at Jimp.<anonymous> (/Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/@jimp/plugin-crop/src/index.ts:73:40)
    at scan (/Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/@jimp/utils/src/index.ts:76:7)
    at Object.crop (/Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/@jimp/plugin-crop/src/index.ts:72:7)
    at Jimp.Jimp.<computed> (/Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/@jimp/core/src/index.ts:200:42)
    at cropAndConvertToDataURL (file:///Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/webdriver-image-comparison/dist/methods/images.js:174:32)
    at async saveWebScreen (file:///Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/webdriver-image-comparison/dist/commands/saveWebScreen.js:59:32)
    at async checkWebScreen (file:///Users/andres.f.losada/Documents/Repositories/videoplayertesting/webdriverIO/node_modules/webdriver-image-comparison/dist/commands/checkWebScreen.js:22:57)
[0-0] 2025-03-18T19:52:24.614Z INFO webdriver: COMMAND deleteSession()

Additional context

My test is a bit more complex, as is runs inside a webview that is embedded within an app, but the issue seems to be only for this plugin. I am using selenium to do a similar test and there is no error. The same happens when trying nightwatch, so I guess there might be a bug in the Jimp implementation that crops the images.

Activity

BosseKarat

BosseKarat commented on Mar 20, 2025

@BosseKarat
Contributor

I am getting the same error using awaitbrowser.emulate('device', 'Galaxy Note 3'); which I'm guessing is easier to debug than setting up a grid.

Here is a minimal example to try. This should fail with a similar error when using chrome or edge.

    it('A sample test', async function () {
        await browser.url('https://webdriver.io/');

        await browser.emulate('device', 'iPhone 15');

        await expect(await browser.checkFullPageScreen('IMG-NAME')).toBeLessThanOrEqual(0.4);
    });

Let me know if the above is a different error and you want me to submit a new ticket.

Edit: I am using latest version of both wdio and visual-serive

wswebcreation

wswebcreation commented on Mar 21, 2025

@wswebcreation
Member

Thanks for adressing this. I'll take a look at it this weekend. I'm also busy with improving this implementation, so it might be that it's already solved in the upcoming release which has a better support for web and hybrid apps

wswebcreation

wswebcreation commented on Mar 22, 2025

@wswebcreation
Member

Hi @AfLosada

I'm trying to debug this, but I don't have a reproducible example. Do you have a sample with a public browser url and an element where you have the same? It might be related to your app, but I want to be sure.

Thanks

wswebcreation

wswebcreation commented on Mar 22, 2025

@wswebcreation
Member

Hmm, this is already what I thought (@BosseKarat , thanks for the simple example). Let me try to explain.

We initially take a screenshot from or a device (@AfLosada your case) or the browser (@BosseKarat your case). We build a new image with Jimp (something like this

const image = await Jimp.read(Buffer.from(newBase64Image, 'base64'));

We then try to crop the element (or something else) out of the image. The crop data in both cases is bigger than the image is, which results in the above error.

Here's an example based on @BosseKarat his test case

// Create an image
const image = await Jimp.read(Buffer.from(newBase64Image, 'base64'));
// Get the dimensions
console.log('Decoded image dimensions:', image.bitmap.width, image.bitmap.height);
// Log: Decoded image dimensions: 1366 629
// The data to crop with
console.log({ x: imageXPosition, y: imageYPosition, w: imageWidth, h: imageHeight })
// Log: { x: 0, y: 0, w: 1365, h: 27180 }
// Now crop
image.crop({ x: imageXPosition, y: imageYPosition, w: imageWidth, h: imageHeight })
// Error
[0-0] RangeError in "@wdio/visual-service desktop.A sample test"
RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 3436852. Received 3436856
    at boundsError (node:internal/buffer:88:9)
    at Buffer.readUInt32BE (node:internal/buffer:311:5)
    at Jimp.<anonymous> (/Users/wimselles/Git/wdio/visual-testing/node_modules/.pnpm/@jimp+plugin-crop@1.6.0/node_modules/@jimp/plugin-crop/src/index.ts:73:40)
    at scan (/Users/wimselles/Git/wdio/visual-testing/node_modules/.pnpm/@jimp+utils@1.6.0/node_modules/@jimp/utils/src/index.ts:76:7)
    at Object.crop (/Users/wimselles/Git/wdio/visual-testing/node_modules/.pnpm/@jimp+plugin-crop@1.6.0/node_modules/@jimp/plugin-crop/src/index.ts:72:7)
    at Jimp.Jimp.<computed> (/Users/wimselles/Git/wdio/visual-testing/node_modules/.pnpm/@jimp+core@1.6.0/node_modules/@jimp/core/src/index.ts:200:42)
    at makeFullPageBase64Image (file:///Users/wimselles/Git/wdio/visual-testing/packages/webdriver-image-comparison/dist/methods/images.js:373:32)
    at async saveFullPageScreen (file:///Users/wimselles/Git/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:72:33)
    at async checkFullPageScreen (file:///Users/wimselles/Git/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/checkFullPageScreen.js:28:44)

@AfLosada , in your case the image might not be fully in the webview. You could validate this
I'm unsure how to solve this if the element image is bigger then the viewport (width or height). I need to think about that

@BosseKarat , in your case the data we get back is way bigger (especially the height). This is already the full viewport height and the module calculations are failing. This will be solved when we support the full BiDi screenshot options, but then you also need to be on the latest version of the webdriverio which supports those options. I need to figure out a way to support that in my module. This is related to #622

BosseKarat

BosseKarat commented on Mar 24, 2025

@BosseKarat
Contributor

Thank you @wswebcreation for the thorough explanation :) I have been using Jimp for many years to stitch together actual+baseline+diff images (stuck on v0.x, v.1.x i broken for me) and now that you mentioned it I kind of remeber that issue. The problem I had was not as advanced as this one but sometimes one of the images came out way bigger than the others due to problems beyond my control. I solved this by calculating the max height of all three images (basically checking which image is tallest) before creating a canvas.

Math.max(actualImg.bitmap.height, baselineImg.bitmap.height, diffImg.bitmap.height);

The output looks like this:

Image

lol, this was way out of topic but I could not help myself once I started thinking about Jimp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @wswebcreation@AfLosada@BosseKarat

      Issue actions

        iOS screenshot error in JIMP when cropping an image received from Appium · Issue #783 · webdriverio/visual-testing