libcamera: ipu3: Always use sensor full frame size

When calculating the pipeline configuration for the IPU3 platform,
libcamera tries to be smart and select the smallest sensor frame
resolution large enough to accommodate the stream sizes
requested by the application.

While this makes a lot of sense, in practice optimizing the
selected sensor resolution makes the pipeline configuration calculation
process fail in multiple occasions, or results in stalls during capture.

As a trivial example, capturing with cam with the following command
line results in a stall:
$ cam -swidth=1280,height=720 -swidth=640,height=480 -c1 -C

Likewise, the Android HAL supported format enumeration fails in
reporting smaller resolutions as supported when used with the OV5670
sensor.

320x240:
DEBUG IPU3 ipu3.cpp:192 CIO2 configuration: 648x486-SGRBG10_IPU3
ERROR IPU3 imgu.cpp:408 Failed to calculate pipe configuration
ERROR IPU3 ipu3.cpp:299 Failed to calculate pipe configuration: unsupported resolutions.

640x480:
DEBUG IPU3 ipu3.cpp:192 CIO2 configuration: 320x240-SGRBG10_IPU3
ERROR IPU3 imgu.cpp:408 Failed to calculate pipe configuration
ERROR IPU3 ipu3.cpp:299 Failed to calculate pipe configuration: unsupported resolutions.

Furthermore the reference xml files used for the IPU3 camera
configuration on the ChromeOS platform restricts the number of sensor
resolution to be used for the OV5670 sensor to 2 from the 6 supported by
the driver [1].

The selection criteria of the correct CIO2 mode are not specified, and
for the time being, as a workaround, always use the sensor maximum
resolution at the expense of frame rate and bus bandwidth to allow the
pipeline to successfully support smaller modes for the OV5670 sensor and
solve pipeline stalls when capturing with both sensors.

[1] See the <sensor_modes> enumeration in:
https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss/graph_settings_ov5670.xml

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>

1 files changed, 13 insertions, 18 deletions

diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp
index 2d881fe2..221259c7 100644
--- a/src/libcamera/pipeline/ipu3/ipu3.cpp
+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp
@@ -144,25 +144,16 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()
 		status = Adjusted;
 	}
 
-	/*
-	 * Validate the requested stream configuration and select the sensor
-	 * format by collecting the maximum RAW stream width and height and
-	 * picking the closest larger match, as the IPU3 can downscale only. If
-	 * no resolution is requested for the RAW stream, use the one from the
-	 * largest YUV stream, plus margins pixels for the IF and BDS to scale.
-	 * If no resolution is requested for any stream, pick the largest one.
-	 */
+	/* Validate the requested stream configuration */
 	unsigned int rawCount = 0;
 	unsigned int yuvCount = 0;
 	Size maxYuvSize;
-	Size maxRawSize;
 
 	for (const StreamConfiguration &cfg : config_) {
 		const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat);
 
 		if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) {
 			rawCount++;
-			maxRawSize.expandTo(cfg.size);
 		} else {
 			yuvCount++;
 			maxYuvSize.expandTo(cfg.size);
@@ -174,18 +165,22 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()
 		return Invalid;
 	}
 
-	if (maxRawSize.isNull())
-		maxRawSize = maxYuvSize.alignedUpTo(IMGU_OUTPUT_WIDTH_MARGIN,
-						    IMGU_OUTPUT_HEIGHT_MARGIN)
-				       .boundedTo(data_->cio2_.sensor()->resolution());
-
 	/*
 	 * Generate raw configuration from CIO2.
 	 *
-	 * The output YUV streams will be limited in size to the maximum
-	 * frame size requested for the RAW stream.
+	 * \todo The image sensor frame size should be selected to optimize
+	 * operations based on the sizes of the requested streams. However such
+	 * a selection makes the pipeline configuration procedure fail for small
+	 * resolutions (for example: 640x480 with OV5670) and causes the capture
+	 * operations to stall for some stream size combinations (see the
+	 * commit message of the patch that introduced this comment for more
+	 * failure examples).
+	 *
+	 * Until the sensor frame size calculation criteria are clarified,
+	 * always use the largest possible one which guarantees better results
+	 * at the expense of the frame rate and CSI-2 bus bandwidth.
 	 */
-	cio2Configuration_ = data_->cio2_.generateConfiguration(maxRawSize);
+	cio2Configuration_ = data_->cio2_.generateConfiguration({});
 	if (!cio2Configuration_.pixelFormat.isValid())
 		return Invalid;