.. SPDX-License-Identifier: CC-BY-SA-4.0 .. section-begin-libcamera =========== libcamera =========== **A complex camera support library for Linux, Android, and ChromeOS** Cameras are complex devices that need heavy hardware image processing operations. Control of the processing is based on advanced algorithms that must run on a programmable processor. This has traditionally been implemented in a dedicated MCU in the camera, but in embedded devices algorithms have been moved to the main CPU to save cost. Blurring the boundary between camera devices and Linux often left the user with no other option than a vendor-specific closed-source solution. To address this problem the Linux media community has very recently started collaboration with the industry to develop a camera stack that will be open-source-friendly while still protecting vendor core IP. libcamera was born out of that collaboration and will offer modern camera support to Linux-based systems, including traditional Linux distributions, ChromeOS and Android. .. section-end-libcamera .. section-begin-getting-started Getting Started --------------- To fetch the sources, build and install: :: git clone https://git.libcamera.org/libcamera/libcamera.git cd libcamera meson build ninja -C build install Dependencies ~~~~~~~~~~~~ The following Debian/Ubuntu packages are required for building libcamera. Other distributions may have differing package names: A C++ toolchain: [required] Either {g++, clang} Meson Build system: [required] meson (>= 0.53) ninja-build pkg-config meson (>= 0.55) is required for building Android (-Dandroid=enabled) If your distribution doesn't provide a recent enough version of meson, you can install or upgrade it using pip3. .. code:: pip3 install --user meson pip3 install --user --upgrade meson for the libcamera core: [required] python3-yaml python3-ply python3-jinja2 for IPA module signing: [required] libgnutls28-dev openssl for improved debugging: [optional] libdw-dev libunwind-dev libdw and libunwind provide backtraces to help debugging assertion failures. Their functions overlap, libdw provides the most detailed information, and libunwind is not needed if both libdw and the glibc backtrace() function are available. for the Raspberry Pi IPA: [optional] libboost-dev Support for Raspberry Pi can be disabled through the meson 'pipelines' option to avoid this dependency. for device hotplug enumeration: [optional] libudev-dev for documentation: [optional] python3-sphinx doxygen graphviz texlive-latex-extra for gstreamer: [optional] libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev for cam: [optional] libevent-dev for qcam: [optional] qtbase5-dev libqt5core5a libqt5gui5 libqt5widgets5 qttools5-dev-tools libtiff-dev for tracing with lttng: [optional] liblttng-ust-dev python3-jinja2 lttng-tools for android: [optional] libexif-dev libjpeg-dev libyaml-dev for lc-compliance: [optional] libevent-dev Using GStreamer plugin ~~~~~~~~~~~~~~~~~~~~~~ To use GStreamer plugin from source tree, set the following environment so that GStreamer can find it. This isn't necessary when libcamera is installed. export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer The debugging tool ``gst-launch-1.0`` can be used to construct a pipeline and test it. The following pipeline will stream from the camera named "Camera 1" onto the OpenGL accelerated display element on your system. .. code:: gst-launch-1.0 libcamerasrc camera-name="Camera 1" ! glimagesink To show the first camera found you can omit the camera-name property, or you can list the cameras and their capabilities using: .. code:: gst-device-monitor-1.0 Video This will also show the supported stream sizes which can be manually selected if desired with a pipeline such as: .. code:: gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1280,height=720' ! \ glimagesink The libcamerasrc element has two log categories, named libcamera-provider (for the video device provider) and libcamerasrc (for the operation of the camera). All corresponding debug messages can be enabled by setting the ``GST_DEBUG`` environment variable to ``libcamera*:7``. Presently, to prevent element negotiation failures it is required to specify the colorimetry and framerate as part of your pipeline construction. For instance, to capture and encode as a JPEG stream and receive on another device the following example could be used as a starting point: .. code:: gst-launch-1.0 libcamerasrc ! \ video/x-raw,colorimetry=bt709,format=NV12,width=1280,height=720,framerate=30/1 ! \ jpegenc ! multipartmux ! \ tcpserversink host=0.0.0.0 port=5000 Which can be received on another device over the network with: .. code:: gst-launch-1.0 tcpclientsrc host=$DEVICE_IP port=5000 ! \ multipartdemux ! jpegdec ! autovideosink .. section-end-getting-started Troubleshooting ~~~~~~~~~~~~~~~ Several users have reported issues with meson installation, crux of the issue is a potential version mismatch between the version that root uses, and the version that the normal user uses. On calling `ninja -C build`, it can't find the build.ninja module. This is a snippet of the error message. :: ninja: Entering directory `build' ninja: error: loading 'build.ninja': No such file or directory This can be solved in two ways: 1) Don't install meson again if it is already installed system-wide. 2) If a version of meson which is different from the system-wide version is already installed, uninstall that meson using pip3, and install again without the --user argument. 'n84' href='#n84'>84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/*
 * Copyright (C) 2019, Google Inc.
 *
 * rkisp1.cpp - RkISP1 Image Processing Algorithms
 */

#include <algorithm>
#include <cstdint>
#include <math.h>
#include <queue>
#include <string.h>

#include <linux/rkisp1-config.h>

#include <ipa/ipa_interface.h>
#include <ipa/ipa_module_info.h>
#include <ipa/rkisp1.h>
#include <libcamera/buffer.h>
#include <libcamera/control_ids.h>
#include <libcamera/request.h>

#include "log.h"
#include "utils.h"

namespace libcamera {

LOG_DEFINE_CATEGORY(IPARkISP1)

class IPARkISP1 : public IPAInterface
{
public:
	int init() override { return 0; }

	void configure(const std::map<unsigned int, IPAStream> &streamConfig,
		       const std::map<unsigned int, ControlInfoMap> &entityControls) override;
	void mapBuffers(const std::vector<IPABuffer> &buffers) override;
	void unmapBuffers(const std::vector<unsigned int> &ids) override;
	void processEvent(const IPAOperationData &event) override;

private:
	void queueRequest(unsigned int frame, rkisp1_isp_params_cfg *params,
			  const ControlList &controls);
	void updateStatistics(unsigned int frame,
			      const rkisp1_stat_buffer *stats);

	void setControls(unsigned int frame);
	void metadataReady(unsigned int frame, unsigned int aeState);

	std::map<unsigned int, BufferMemory> bufferInfo_;

	ControlInfoMap ctrls_;

	/* Camera sensor controls. */
	bool autoExposure_;
	uint32_t exposure_;
	uint32_t minExposure_;
	uint32_t maxExposure_;
	uint32_t gain_;
	uint32_t minGain_;
	uint32_t maxGain_;
};

void IPARkISP1::configure(const std::map<unsigned int, IPAStream> &streamConfig,
			  const std::map<unsigned int, ControlInfoMap> &entityControls)
{
	if (entityControls.empty())
		return;

	ctrls_ = entityControls.at(0);

	const auto itExp = ctrls_.find(V4L2_CID_EXPOSURE);
	if (itExp == ctrls_.end()) {
		LOG(IPARkISP1, Error) << "Can't find exposure control";
		return;
	}

	const auto itGain = ctrls_.find(V4L2_CID_ANALOGUE_GAIN);
	if (itGain == ctrls_.end()) {
		LOG(IPARkISP1, Error) << "Can't find gain control";
		return;
	}

	autoExposure_ = true;

	minExposure_ = std::max<uint32_t>(itExp->second.min().get<int32_t>(), 1);
	maxExposure_ = itExp->second.max().get<int32_t>();
	exposure_ = minExposure_;

	minGain_ = std::max<uint32_t>(itGain->second.min().get<int32_t>(), 1);
	maxGain_ = itGain->second.max().get<int32_t>();
	gain_ = minGain_;

	LOG(IPARkISP1, Info)
		<< "Exposure: " << minExposure_ << "-" << maxExposure_
		<< " Gain: " << minGain_ << "-" << maxGain_;

	setControls(0);
}

void IPARkISP1::mapBuffers(const std::vector<IPABuffer> &buffers)
{
	for (const IPABuffer &buffer : buffers) {
		bufferInfo_[buffer.id] = buffer.memory;
		bufferInfo_[buffer.id].planes()[0].mem();
	}
}

void IPARkISP1::unmapBuffers(const std::vector<unsigned int> &ids)
{
	for (unsigned int id : ids)
		bufferInfo_.erase(id);
}

void IPARkISP1::processEvent(const IPAOperationData &event)
{
	switch (event.operation) {
	case RKISP1_IPA_EVENT_SIGNAL_STAT_BUFFER: {
		unsigned int frame = event.data[0];
		unsigned int bufferId = event.data[1];

		const rkisp1_stat_buffer *stats =
			static_cast<rkisp1_stat_buffer *>(bufferInfo_[bufferId].planes()[0].mem());

		updateStatistics(frame, stats);
		break;
	}
	case RKISP1_IPA_EVENT_QUEUE_REQUEST: {
		unsigned int frame = event.data[0];
		unsigned int bufferId = event.data[1];

		rkisp1_isp_params_cfg *params =
			static_cast<rkisp1_isp_params_cfg *>(bufferInfo_[bufferId].planes()[0].mem());

		queueRequest(frame, params, event.controls[0]);
		break;
	}
	default:
		LOG(IPARkISP1, Error) << "Unkown event " << event.operation;
		break;
	}
}

void IPARkISP1::queueRequest(unsigned int frame, rkisp1_isp_params_cfg *params,
			     const ControlList &controls)
{
	/* Prepare parameters buffer. */
	memset(params, 0, sizeof(*params));

	/* Auto Exposure on/off. */
	if (controls.contains(controls::AeEnable)) {
		autoExposure_ = controls.get(controls::AeEnable);
		if (autoExposure_)
			params->module_ens = CIFISP_MODULE_AEC;

		params->module_en_update = CIFISP_MODULE_AEC;
	}

	IPAOperationData op;
	op.operation = RKISP1_IPA_ACTION_PARAM_FILLED;

	queueFrameAction.emit(frame, op);
}