Documentation/sensor_driver_requirements.rst


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

.. SPDX-License-Identifier: CC-BY-SA-4.0

.. _sensor-driver-requirements:

Sensor Driver Requirements
==========================

libcamera handles imaging devices in the CameraSensor class and defines
a consistent interface through its API towards other library components.

The CameraSensor class uses the V4L2 subdev kernel API to interface with the
camera sensor through one or multiple sub-devices exposed in userspace by
the sensor driver.

In order for libcamera to be fully operational and provide all the required
information to interface with the camera sensor to applications and pipeline
handlers, a set of mandatory and optional features the driver has to support
has been defined.

Mandatory Requirements
----------------------

The sensor driver is assumed to be fully compliant with the V4L2 specification.

For RAW sensors, the sensor driver shall support the following V4L2 controls:

* `V4L2_CID_EXPOSURE`_
* `V4L2_CID_HBLANK`_
* `V4L2_CID_PIXEL_RATE`_
* `V4L2_CID_VBLANK`_

.. _V4L2_CID_EXPOSURE: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html
.. _V4L2_CID_HBLANK: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-image-source.html
.. _V4L2_CID_PIXEL_RATE: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-image-process.html
.. _V4L2_CID_VBLANK: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-image-source.html

While V4L2 doesn't specify a unit for the ``EXPOSURE`` control, libcamera
requires it to be expressed as a number of image lines. Camera sensor drivers
that do not comply with this requirement will need to be adapted or will produce
incorrect results.

The ``HBLANK``, ``PIXEL_RATE`` and ``VBLANK`` controls are used to compute the
sensor output timings.

Optional Requirements
---------------------

The sensor driver should support the following V4L2 controls:

* `V4L2_CID_CAMERA_ORIENTATION`_
* `V4L2_CID_CAMERA_SENSOR_ROTATION`_

.. _V4L2_CID_CAMERA_ORIENTATION: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html
.. _V4L2_CID_CAMERA_SENSOR_ROTATION: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html

The controls are used to register the camera location and rotation.

The sensor driver should implement support for the V4L2 Selection API,
specifically it should implement support for the
`VIDIOC_SUBDEV_G_SELECTION`_ ioctl with support for the following selection
targets:

.. _VIDIOC_SUBDEV_G_SELECTION: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/vidioc-subdev-g-selection.html#c.V4L.VIDIOC_SUBDEV_G_SELECTION

* `V4L2_SEL_TGT_CROP_BOUNDS`_ to report the readable pixel array area size
* `V4L2_SEL_TGT_CROP_DEFAULT`_ to report the active pixel array area size
* `V4L2_SEL_TGT_CROP`_ to report the analogue selection rectangle

Support for the selection API is scheduled to become a mandatory feature in
the near future.

.. _V4L2_SEL_TGT_CROP_BOUNDS: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2-selection-targets.html
.. _V4L2_SEL_TGT_CROP_DEFAULT: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2-selection-targets.html
.. _V4L2_SEL_TGT_CROP: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2-selection-targets.html
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
 * Copyright (C) 2021, Ideas on Board Oy
 *
 * KMS Sink
 */

#include "kms_sink.h"

#include <array>
#include <algorithm>
#include <assert.h>
#include <iostream>
#include <limits.h>
#include <memory>
#include <stdint.h>
#include <string.h>

#include <libcamera/camera.h>
#include <libcamera/formats.h>
#include <libcamera/framebuffer.h>
#include <libcamera/stream.h>

#include "drm.h"

KMSSink::KMSSink(const std::string &connectorName)
	: connector_(nullptr), crtc_(nullptr), plane_(nullptr), mode_(nullptr)
{
	int ret = dev_.init();
	if (ret < 0)
		return;

	/*
	 * Find the requested connector. If no specific connector is requested,
	 * pick the first connected connector or, if no connector is connected,
	 * the first connector with unknown status.
	 */
	for (const DRM::Connector &conn : dev_.connectors()) {
		if (!connectorName.empty()) {
			if (conn.name() != connectorName)
				continue;

			connector_ = &conn;
			break;
		}

		if (conn.status() == DRM::Connector::Connected) {
			connector_ = &conn;
			break;
		}

		if (!connector_ && conn.status() == DRM::Connector::Unknown)
			connector_ = &conn;
	}

	if (!connector_) {
		if (!connectorName.empty())
			std::cerr
				<< "Connector " << connectorName << " not found"
				<< std::endl;
		else
			std::cerr << "No connected connector found" << std::endl;
		return;
	}

	dev_.requestComplete.connect(this, &KMSSink::requestComplete);
}

void KMSSink::mapBuffer(libcamera::FrameBuffer *buffer)
{
	std::array<uint32_t, 4> strides = {};

	/* \todo Should libcamera report per-plane strides ? */
	unsigned int uvStrideMultiplier;

	switch (format_) {
	case libcamera::formats::NV24:
	case libcamera::formats::NV42:
		uvStrideMultiplier = 4;
		break;
	case libcamera::formats::YUV420:
	case libcamera::formats::YVU420:
	case libcamera::formats::YUV422:
		uvStrideMultiplier = 1;
		break;
	default:
		uvStrideMultiplier = 2;
		break;
	}

	strides[0] = stride_;
	for (unsigned int i = 1; i < buffer->planes().size(); ++i)
		strides[i] = stride_ * uvStrideMultiplier / 2;

	std::unique_ptr<DRM::FrameBuffer> drmBuffer =
		dev_.createFrameBuffer(*buffer, format_, size_, strides);
	if (!drmBuffer)
		return;

	buffers_.emplace(std::piecewise_construct,
			 std::forward_as_tuple(buffer),
			 std::forward_as_tuple(std::move(drmBuffer)));
}

int KMSSink::configure(const libcamera::CameraConfiguration &config)
{
	if (!connector_)
		return -EINVAL;

	crtc_ = nullptr;
	plane_ = nullptr;
	mode_ = nullptr;

	const libcamera::StreamConfiguration &cfg = config.at(0);

	/* Find the best mode for the stream size. */
	const std::vector<DRM::Mode> &modes = connector_->modes();

	unsigned int cfgArea = cfg.size.width * cfg.size.height;
	unsigned int bestDistance = UINT_MAX;

	for (const DRM::Mode &mode : modes) {
		unsigned int modeArea = mode.hdisplay * mode.vdisplay;
		unsigned int distance = modeArea > cfgArea ? modeArea - cfgArea
				      : cfgArea - modeArea;

		if (distance < bestDistance) {
			mode_ = &mode;
			bestDistance = distance;

			/*
			 * If the sizes match exactly, there will be no better
			 * match.
			 */
			if (distance == 0)
				break;
		}
	}

	if (!mode_) {
		std::cerr << "No modes\n";
		return -EINVAL;
	}

	int ret = configurePipeline(cfg.pixelFormat);
	if (ret < 0)
		return ret;

	size_ = cfg.size;
	stride_ = cfg.stride;

	/* Configure color space. */
	colorEncoding_ = std::nullopt;
	colorRange_ = std::nullopt;

	if (cfg.colorSpace->ycbcrEncoding == libcamera::ColorSpace::YcbcrEncoding::None)
		return 0;

	/*
	 * The encoding and range enums are defined in the kernel but not
	 * exposed in public headers.
	 */
	enum drm_color_encoding {
		DRM_COLOR_YCBCR_BT601,
		DRM_COLOR_YCBCR_BT709,
		DRM_COLOR_YCBCR_BT2020,
	};

	enum drm_color_range {
		DRM_COLOR_YCBCR_LIMITED_RANGE,
		DRM_COLOR_YCBCR_FULL_RANGE,
	};

	const DRM::Property *colorEncoding = plane_->property("COLOR_ENCODING");
	const DRM::Property *colorRange = plane_->property("COLOR_RANGE");

	if (colorEncoding) {
		drm_color_encoding encoding;

		switch (cfg.colorSpace->ycbcrEncoding) {
		case libcamera::ColorSpace::YcbcrEncoding::Rec601:
		default:
			encoding = DRM_COLOR_YCBCR_BT601;
			break;
		case libcamera::ColorSpace::YcbcrEncoding::Rec709:
			encoding = DRM_COLOR_YCBCR_BT709;
			break;
		case libcamera::ColorSpace::YcbcrEncoding::Rec2020:
			encoding = DRM_COLOR_YCBCR_BT2020;
			break;
		}

		for (const auto &[id, name] : colorEncoding->enums()) {
			if (id == encoding) {
				colorEncoding_ = encoding;
				break;
			}
		}
	}

	if (colorRange) {
		drm_color_range range;

		switch (cfg.colorSpace->range) {
		case libcamera::ColorSpace::Range::Limited:
		default:
			range = DRM_COLOR_YCBCR_LIMITED_RANGE;
			break;
		case libcamera::ColorSpace::Range::Full:
			range = DRM_COLOR_YCBCR_FULL_RANGE;
			break;
		}

		for (const auto &[id, name] : colorRange->enums()) {
			if (id == range) {
				colorRange_ = range;
				break;
			}
		}
	}

	if (!colorEncoding_ || !colorRange_)
		std::cerr << "Color space " << cfg.colorSpace->toString()
			  << " not supported by the display device."
			  << " Colors may be wrong." << std::endl;

	return 0;
}

int KMSSink::selectPipeline(const libcamera::PixelFormat &format)
{
	/*
	 * If the requested format has an alpha channel, also consider the X
	 * variant.
	 */
	libcamera::PixelFormat xFormat;

	switch (format) {
	case libcamera::formats::ABGR8888:
		xFormat = libcamera::formats::XBGR8888;
		break;
	case libcamera::formats::ARGB8888:
		xFormat = libcamera::formats::XRGB8888;
		break;
	case libcamera::formats::BGRA8888:
		xFormat = libcamera::formats::BGRX8888;
		break;
	case libcamera::formats::RGBA8888:
		xFormat = libcamera::formats::RGBX8888;
		break;
	}

	/*
	 * Find a CRTC and plane suitable for the request format and the
	 * connector at the end of the pipeline. Restrict the search to primary
	 * planes for now.
	 */
	for (const DRM::Encoder *encoder : connector_->encoders()) {
		for (const DRM::Crtc *crtc : encoder->possibleCrtcs()) {
			for (const DRM::Plane *plane : crtc->planes()) {
				if (plane->type() != DRM::Plane::TypePrimary)
					continue;

				if (plane->supportsFormat(format)) {
					crtc_ = crtc;
					plane_ = plane;
					format_ = format;
					return 0;
				}

				if (plane->supportsFormat(xFormat)) {
					crtc_ = crtc;
					plane_ = plane;
					format_ = xFormat;
					return 0;
				}
			}
		}
	}

	return -EPIPE;
}

int KMSSink::configurePipeline(const libcamera::PixelFormat &format)
{
	const int ret = selectPipeline(format);
	if (ret) {
		std::cerr
			<< "Unable to find display pipeline for format "