IMX708: Add support for 4-lane operation #6775

peyton-howe · 2025-04-09T09:13:44Z

This is just to get the ball rolling, will make sure to cleanup commits, descriptions, and break up the dtoverlay and driver into separate PRs.

I successfully have 4-lane operation automatically set by the overlay option inside the driver, dynamically switching the PLL registers as needed! I have tested max resolution @ 30fps and haven't run into any issues so far. rpicam-hello shows the following framerates and resolutions

1536x864 [125.69 fps - (768, 432)/3072x1728 crop]
2304x1296 [86.63 fps - (0, 0)/4608x2592 crop]
4608x2592 [31.60 fps - (0, 0)/4608x2592 crop]

Is 30fps the upper limit for max resolution on the RPI? I based the settings off what looks to be a 30fps mode for the X2 phone, but this datasheet indicates the max resolution of the sensor is 42fps in 10bit mode.

As for the other the 2304x1296 mode, I just followed 6by9's suggestion of halving the line length paired with the higher link frequency. Maybe more performance can be squeezed out of this one by tuning the PLL settings?

…e number of MIPI lanes

…ngs based on the number of mipi lanes

…ming convention

pelwell · 2025-04-09T10:05:23Z

Leave another comment when you think it is worth rerunning the build checks.

6by9

I'll have a more detailed look later.

The datasheet you link to is for IMX 078, not 708.

The main datasheet says max frame rate at full res if 60fps, but that will be at 2.5Gb/s per lane which no Pi supports. 30fps for your 1094Mbit/s data rate seems plausible (just).

Forcing the use of 547MHz link frequency in 4 lane mode is a little annoying. Although dtoverlay=imx708,4lane,link-frequency=450000000 would drop it back down. Does there need to be validation of 2 lane vs 4 lane link frequencies?

6by9 · 2025-04-10T19:10:33Z

drivers/media/i2c/imx708.c

 	{0x3062, 0x00},
-	{0x3063, 0x12},
+	{0x3063, 0x12}, //0x30, 0x12


You don't want to be changing this register.

peyton-howe · 2025-04-11T17:16:46Z

I dropped the link frequency back to 450000000 and set the pixel rate to 777.6Mhz and I was able to get a stable output of 28.13fps. I also tried increasing the pixel clock to get closer to 30fps, but it just resulted in artificating and other sensor errors - I tried pixel rates of 854.4Mhz and 801.6Mhz.

I've also tried dropping the line length down to 5216 and tried all the current pixel rates in the driver but I just get artifcating and this libcamera error (still at the 450000000 link frequency):

Sensor minimum line length of 9.21us (500.378 MPix/s) is below the minimum allowable ISP limit of 12.13us (380 MPix/s)

6by9 · 2025-04-16T11:11:00Z

Sensor minimum line length of 9.21us (500.378 MPix/s) is below the minimum allowable ISP limit of 12.13us (380 MPix/s)

will always result in corruption as the CFE overflows.

IMX708 is the first sensor where there have been such distinct line length values that could be supported. Unfortunately it means that the pixel rate and HBLANKing values do need to allow for sufficient time per line.

It's curious that the IVT multiplier is different between different modes of the sensor. It just makes the driver a little more complicated. I can't think of an obvious reason why the highest value can't be used for all modes, but I haven't looked at the full timings to ensure that it can offload the image data over CSI2 within the relevant line period.

There must be a setting of IVT that means the line length of 5216 comes in at >=12.13us

I'm afraid I simply don't have time to play with imx708 at present.
I'd be happy with the simple patch that added 4 lane mode in with the same timings. Likewise adding extra link frequencies. Trying to handle changing pixel rates and HBLANK is going to be more significant, and the risk of regressions will be high.

peyton-howe · 2025-05-06T08:38:06Z

I've successfully been able to get it running at a line length of 5216, but I still run into the limit of >=12.13us for the line length. Anything below that and libcamera complains and the image output is all messed up. The highest I've been able to push the camera is 31fps with a line length of 5216 and a pixel rate of 432Mhz. This combo throws an error in IPARPI and messes with the frame rate calculation, but doesn't cause any visible artifcating on the sensor.

Based on my calculations for the bandwith of the pi5, we should be able to get ~35fps, but I'm not seeing any way around this 12.13us line length limit as I'd need to push the pixel rate higher (which lowers the line length) in order to squeeze out the last few fps. Do you have any other ideas?

6by9 · 2025-05-07T18:06:09Z

As it appears there are such restrictions on the line_length register for IMX708, the only route I can see would be to alter the pixel rate so that the line takes 12.13us to readout with whichever line length setting most sensibly matches that. The max frame rate will then be dictated by the number of lines required in your image.

There is an option to overclock RP1 to increase the throughput of the CFE slightly. Those working with imx294 have been doing that - see #6725 and raspberrypi/libcamera#243

peyton-howe · 2025-05-15T06:21:37Z

That did the trick! I didn't realize that the RP1 was limiting the throughput to 380Mpix/s.

I overclocked it to 300Mhz as outlined in those PRs, and also overclocked the mipi lanes to 1800Mhz and I'm able to get a stable 45fps output! I think this is pretty much the upper limit we can do on the pi5, unless I can overclock the mipi lanes even higher?

phansel · 2025-05-18T03:51:21Z

Hopefully this isn't too off-topic of a question. Has anyone been able to run the IMX708 in a single-lane configuration?

ed: I suppose it would just be a change to the line "data-lanes = <1 2 3 4>;", and probably IMX708_CSI_2_LANE_MODE = 0x00?

6by9 · 2025-05-19T10:00:14Z

Hopefully this isn't too off-topic of a question. Has anyone been able to run the IMX708 in a single-lane configuration?

ed: I suppose it would just be a change to the line "data-lanes = <1 2 3 4>;", and probably IMX708_CSI_2_LANE_MODE = 0x00?

Not supported.
Register 0x0114 bits [1:0] lists values 0 and 2 (for 1 and 3 lanes respectively) as "not supported" with the comment "This sensor supports only 2Lane and 4Lane".

phansel · 2025-05-19T19:30:26Z

Not supported. Register 0x0114 bits [1:0] lists values 0 and 2 (for 1 and 3 lanes respectively) as "not supported" with the comment "This sensor supports only 2Lane and 4Lane".

That's disappointing. Thanks for the definitive answer.

rajeshpoddar · 2025-05-29T20:30:28Z

also overclocked the mipi lanes to 1800Mhz

How did you do that? In dphy.c I only see options for going up to 1500Mhz -

linux/drivers/media/platform/raspberrypi/rp1_cfe/dphy.c

Line 99 in 23e6672

static void dphy_set_hsfreqrange(struct dphy_data *dphy, uint32_t mbps)

.

* Dynamically calculate link frequency registers * Add more pixel rates for testing * Drop line_length_pix to the lowest value for the resolution and mode * Initial attempt to dynamically calculate the pixel rate

peyton-howe · 2025-06-06T07:51:35Z

@rajeshpoddar I just set dtoverlay=imx708,link-frequency=900000000 in /boot/firmware/config.txt

peyton-howe · 2025-06-06T08:05:31Z

@6by9 I just pushed a commit that will dynamically calculate the link frequency registers. I'm also trying to dynamically set the pixel rate by setting the line length to the minimum value possible for the resolution/mode and then calculating the appropriate pixel rate based on the link frequency and number of mipi lanes.

I want to make sure I handle this calculation properly, so I've got a couple of questions.

Am I correct to assume that the data is being sent out as packed 8b data on the Pi5 or packed MIPI CSI2 for the Pi4 and below since the mode shows SRGGB10_CSI2P in rpicam-apps? If so, is the 1byte/1.25bytes per pixel what I need to use for calculating the appropriate pixel rate?
Should I use vblank_min or vblank_default in my calculation?
My approach was going to be push the pixel rate to the max it can be for the given link frequency and mipi lanes. What are your concerns with this?

njhollinghurst · 2025-07-31T17:40:48Z

drivers/media/i2c/imx708.c

@@ -792,7 +846,7 @@ static const struct imx708_mode supported_modes_10bit_no_hdr[] = {
 			.num_of_regs = ARRAY_SIZE(mode_4608x2592_regs),
 			.regs = mode_4608x2592_regs,
 		},
-		.pixel_rate = {595200000, 854400000},
+		.pixel_rate = {198400000, 304000000},


I'm curious about this factor-of-3 change of pixel rate and line length (though I never understood the rationale of our original settings either). Is it a real, functional change or just a change in the way things are represented? Does it improve image quality?

It shouldn't change image quality at all, only readout rate.

I suspect that we just copied the values provided by Sony in their big spreadsheet as supposedly the optimised values, but they often wrote the register set based on a requested resolution/frame rate combination whereas we actually want the max that can be achieved by the hardware.

Max pixel rate is going to depend on the link frequency to ensure we can output all the pixel data in the appropriate period of time. I can see a case where the driver ought to verify that the selected link frequency can do that, because otherwise DT can give us unworkable configurations. This is going to need a fair amount more analysis.

Edit: Specifically for full-res mode and with 2 lanes, the old rate (595200000) was the Sony recommended value. It has suddenly been divided by 3 (and the line width changed to match). I just wondered why this was changed?

I dropped the line length down to their minimum possible values for the given resolution, as outlined here. I plan to clean this up so there's only a single entry for line_length_pix, like before. This will ensure the user is always able to set the maximum framerate based on the number of MIPI lanes and the data rate of those lanes.

I then adjusted the pixel rate accordingly for the 2-lane full-res mode to match the original settings with the lower line length.

Fair enough. I don't know why Sony recommended the faster rate (and they might not be willing to explain). I thought it might be extra cycles needed for re-Mosaic but if the image is unaffected then perhaps it's not needed.

njhollinghurst · 2025-08-04T13:20:32Z

drivers/media/i2c/imx708.c

+	IMX708_LINK_FREQ_547MHZ,
+	IMX708_LINK_FREQ_640MHZ,
+	IMX708_LINK_FREQ_750MHZ,
+	IMX708_LINK_FREQ_900MHZ,


A tangential suggestion: Do we even need discrete values for link frequency: Couldn't we just support all frequencies from 447MHz to 750MHz, in 3MHz steps, if that is what the PLL is capable of?

Of course the default values in DeviceTree will have to be carefully chosen for compatibility and EMC.

The PLL should be capable of even more than that as both IVT_PREPLLCK_DIV and IVT_PLL_MPY could be modified. Potentially the ccs-pll helper functions could compute PLL settings, but it adds quite a lot of complexity.

You actually get more fun in having to create the array of s64's for the V4L2_CID_LINK_FREQUENCY control - probably stick it in struct imx708 so that every instance of imx708 could choose a preferred link frequency.

Default values would remain as is for 2 lane.
4 lane we probably declare as experimental and not EMC tested.

I don't think we need that flexibility -- multiples of 3MHz seem fine -- I wondered if we could get rid of this apparently random-looking list of values, and replace it by a stepped range?

Actually some of these magic-looking numbers are not multiples of 3MHz (547MHz and 640MHz). Are those implemented accurately right now?

I am experimenting at the moment with other clock frequencies like 480MHz and it feels silly to have to add them to the enum just to try them out.

The magic should be:
(24MHz / (reg 0x030d)) * ((reg 0x030e<<8) + (reg 0x030f)) / 2
0x030d is 4 in all modes, giving us 3MHz * (030e/f)

I'd agree that imx708_configure_link_frequency is computing these values, but there is no point in having the enum. Read the raw link frequency from DT, validate it against selected clock frequency and reg 0x030d, and store that value in struct imx708.
Actually the LINK_FREQUENCY control can point to that as the single array value, and imx708_configure_link_frequency can use it to compute the register settings.

njhollinghurst · 2025-08-04T13:32:45Z

arch/arm/boot/dts/overlays/imx708-overlay.dts

@@ -95,6 +111,7 @@
 		vcm = <&vcm_node>, "status",
 		      <0>, "=4";
 		link-frequency = <&cam_endpoint>,"link-frequencies#0";
+		4lane = <0>, "+201+202";


It looks like there could be a confusing interaction between link-frequency and 4lane overrides, which both try to change <&cam_endpoint>/link-frequencies. The actual frequency may depend on the order in which the dtparams are given.

If that's the case, is there a cleaner way to do it that would always give priority to the link-frequency?

Sadly there isn't a way to set priorities on DT overrides.

What could be done is to add a fragment 203 which sets the link frequency for 2 lane mode. link-frequency then updates both 202 and 203, and 4lane disables 203 as well as enabling 201 and 202. At least that should give a consistent behaviour.

njhollinghurst · 2025-08-04T15:06:55Z

drivers/media/i2c/imx708.c

@@ -689,7 +846,7 @@ static const struct imx708_mode supported_modes_10bit_no_hdr[] = {
 			.num_of_regs = ARRAY_SIZE(mode_4608x2592_regs),
 			.regs = mode_4608x2592_regs,
 		},
-		.pixel_rate = 595200000,
+		.pixel_rate = {198400000, 304000000},


I've tested with 4 lanes your suggested pixel rate of 416000000 for 30fps, and it seems to work fine -- not only at a MIPI link frequency of "547MHz" (which I suspect is actually 546MHz) with no need to overclock RP1; but also at the perhaps more Pi-friendly 492MHz and as low as 480MHz (though not, alas, at 450MHz).

So I would be happy to see 30fps supported here. Of course it depends on the more general question of default link frequency.

peyton-howe added 6 commits April 2, 2025 04:00

imx708: intial support for 4-lane operation

2d38887

Update pixel rate for 4lane full resolution and make HBLANK controllable

0a886f7

Rework imx708 driver to modify pixel rate and line length based on th…

2e4be19

…e number of MIPI lanes

Some minor cleanup to better match IMX415 structure for dynamic setti…

9e77b99

…ngs based on the number of mipi lanes

Add missing vblank back to set_framing_limits and minor cleanup of na…

1fd4408

…ming convention

Back to unsigned int for line_length_pix

f31864a

6by9 reviewed Apr 10, 2025

View reviewed changes

Restructure WIP

55c7b20

* Dynamically calculate link frequency registers * Add more pixel rates for testing * Drop line_length_pix to the lowest value for the resolution and mode * Initial attempt to dynamically calculate the pixel rate

njhollinghurst reviewed Jul 31, 2025

View reviewed changes

njhollinghurst reviewed Aug 4, 2025

View reviewed changes

IMX708: Add support for 4-lane operation #6775

Are you sure you want to change the base?

IMX708: Add support for 4-lane operation #6775

Conversation

peyton-howe commented Apr 9, 2025

Uh oh!

pelwell commented Apr 9, 2025

Uh oh!

6by9 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peyton-howe commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

6by9 commented Apr 16, 2025

Uh oh!

peyton-howe commented May 6, 2025

Uh oh!

6by9 commented May 7, 2025

Uh oh!

peyton-howe commented May 15, 2025

Uh oh!

phansel commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

6by9 commented May 19, 2025

Uh oh!

phansel commented May 19, 2025

Uh oh!

rajeshpoddar commented May 29, 2025

Uh oh!

peyton-howe commented Jun 6, 2025

Uh oh!

peyton-howe commented Jun 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

njhollinghurst Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

njhollinghurst Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peyton-howe commented Apr 11, 2025 •

edited

Loading

phansel commented May 18, 2025 •

edited

Loading

njhollinghurst Aug 4, 2025 •

edited

Loading

njhollinghurst Aug 4, 2025 •

edited

Loading