libcamera: IPAInterface: Replace C API with the new C++-only API

Remove everything related to the C API, including ipa_context,
ipa_context_wrapper, and IPAInterfaceWrapper. Also remove relevant
documentation.

ipaCreate() provided by IPA implementations, and createInterface()
provided by IPAModule (wrapper around IPA implementation) both now
return a C++ object IPAInterface instead of struct ipa_context.

Although IPAInterfaceWrapper is the only component of libipa, the
skeleton and build files for libipa are retained.

After converting the C API to the C++-only API, make all pipeline
handlers and IPAs use the new API.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

This is a combination of 21 commits:

---

libcamera: IPAModule: Replace ipa_context with IPAInterface

With the new IPC infrastructure, we no longer need the C interface as
provided by struct ipa_context. Make ipaCreate_() and createInterface()
return IPAInterface.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: ipa_context_wrapper: Remove ipa_context_wrapper

Since ipa_context has been replaced with custom IPAInterfaces, it is not
longer needed. Remove it.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: IPAInterface: remove ipa_context and functions from documentation

Remove all the documentation related to ipa_context and the C IPA API,
as well as the documentation about the functions in the IPAInterface.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

---

libcamera: IPAInterface: Remove all functions from IPAInterface

Now that all the functions in the IPA interface are defined in the data
definition file and a specialized IPAInterface is generated per pipeline
handler, remove all the functions from the base IPAInterface.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: IPAInterface: make ipaCreate return IPAInterface

With the new IPC infrastructure, we no longer need the C interface as
provided by struct ipa_context. Make ipaCreate return IPAinterface.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

---

ipa: remove IPAInterfaceWrapper

As every pipeline has its own proxy, IPAInterfaceWrapper is no
longer necessary. Remove it.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Acked-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

---

libcamera: IPAProxy: Remove stop() override

Since stop() is part of the IPA interface, and the IPA interface is now
generated based on the data definition file per pipeline, this no longer
needs to be overrided by the base IPAProxy. Remove it.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: IPAProxy, IPAManager: Switch to one-proxy-per-pipeline scheme

IPAProxy is changed in two major ways:
- Every pipeline has its own proxy, to support each pipeline's IPA
  interface
- IPAProxy implementations always encapsulate IPA modules, and switch
  internally for isolation or threaded

The IPAProxy registration mechanism is removed, as each pipeline will
have its own proxy, so the pipeline can pass the specialized class name
of the IPAProxy to the IPAManager for construction.

IPAManager is changed accordingly to support these changes:
- createIPA is a template function that takes an IPAProxy class, and
  always returns an IPAProxy
- IPAManager no longer decides on isolation, and simply creates an
  IPAProxy instance while passing the isolation flag

Consequently, the old IPAProxy classes (IPAProxyThread and
IPAProxyLinux) are removed. The IPAInterfaceTest is updated to use
the new IPAManager interface, and to construct a ProcessManager as no
single global instance is created anymore.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>

---

libcamera: IPAProxy: Add isolate parameter to create()

Since IPAProxy implementations now always encapsulate IPA modules, add a
parameter to create() to signal if the proxy should isolate the IPA or not.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline

Now that each pipeline handler has its own IPAProxy implementation, make
the IPAManager fetch the IPAProxy based on the pipeline handler name.
Also, since the IPAProxy is used regardless of isolation or no
isolation, remove the isolation check from the proxy selection.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

---

libcamera: IPAManager: add isolation flag to proxy creation

When the IPA proxy is created, it needs to know whether to isolate or
not. Feed the flag at creation of the IPA proxy.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: IPAManager: Make createIPA return proxy directly

Since every pipeline knows the type of the proxy that it needs, and
since all IPAs are to be wrapped in a proxy, IPAManager no longer needs
to search in the factory list to fetch the proxy factory to construct a
factory. Instead, we define createIPA as a template function, and the
pipeline handler can declare the proxy type when it calls createIPA.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: IPAProxy: Remove registration mechanism

Implementations of IPA proxies use a registration mechanism to register
themselves with the main IPA proxy factory. This registration declares
static objects, causing a risk of things being constructed before the
proper libcamera facilities are ready. Since each pipeline handler has
its own IPA proxy and knows the type, it isn't necessary to have a proxy
factory. Remove it to alleviate the risk of early construction.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread

We have now changed the proxy from per-IPC mechanism to per-pipeline.
The per-IPC mechanism proxies are thus no longer needed; remove them.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

tests: ipa_interface_test: Update to use new createIPA

Update the IPA interface test to use the new createIPA function from
IPAManager. Also create an instance of ProcessManager, as no single
global instance is created automatically anymore. Update meson.build to
to depend on the generated IPA interface headers.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: PipelineHandler: Remove IPA from base class

Since pipeline handlers now have their own IPA interface types, it can no
longer be defined in the base class, and each pipeline handler
implementation must declare it and its type themselves. Remove it from
the base class.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

ipa: raspberrypi: Add mojom data definition file

Add a mojom data definition for raspberrypi pipeline handler's IPAs.
This simplifies the API between the raspberrypi pipeline handler and the
IPA, and is not a direct translation of what was used before with
IPAOperationData.

Also move the enums from raspberrypi.h to raspberrypi.mojom

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: pipeline, ipa: raspberrypi: Use new data definition

Now that we can generate custom functions and data structures with mojo,
switch the raspberrypi pipeline handler and IPA to use the custom data
structures as defined in the mojom data definition file.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: pipeline, ipa: vimc: Support the new IPC mechanism

Add support to vimc pipeline handler and IPA for the new IPC mechanism.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

---

libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism

Add support to the rkisp1 pipeline handler and IPA for the new IPC
mechanism.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

---

libcamera: pipeline, ipa: ipu3: Support the new IPC mechanism

Add support to ipu3 pipeline handler and IPA for the new IPC mechanism.

[Original version]
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
[Fixed commit message and small changes]
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

1 files changed, 0 insertions, 298 deletions

diff --git a/src/libcamera/ipa_context_wrapper.cpp b/src/libcamera/ipa_context_wrapper.cpp
deleted file mode 100644
index 19c44ad8..00000000
--- a/src/libcamera/ipa_context_wrapper.cpp
+++ /dev/null
@@ -1,298 +0,0 @@
-/* SPDX-License-Identifier: LGPL-2.1-or-later */
-/*
- * Copyright (C) 2019, Google Inc.
- *
- * ipa_context_wrapper.cpp - Image Processing Algorithm context wrapper
- */
-
-#include "libcamera/internal/ipa_context_wrapper.h"
-
-#include <vector>
-
-#include <libcamera/controls.h>
-
-#include "libcamera/internal/byte_stream_buffer.h"
-#include "libcamera/internal/camera_sensor.h"
-#include "libcamera/internal/utils.h"
-
-/**
- * \file ipa_context_wrapper.h
- * \brief Image Processing Algorithm context wrapper
- */
-
-namespace libcamera {
-
-/**
- * \class IPAContextWrapper
- * \brief Wrap an ipa_context and expose it as an IPAInterface
- *
- * The IPAContextWrapper class wraps an ipa_context, provided by an IPA module, and
- * exposes an IPAInterface. This mechanism is used for IPAs that are not
- * isolated in a separate process to allow direct calls from pipeline handler
- * using the IPAInterface API instead of the lower-level ipa_context API.
- *
- * The IPAInterface methods are converted to the ipa_context API by translating
- * all C++ arguments into plain C structures or byte arrays that contain no
- * pointer, as required by the ipa_context API.
- */
-
-/**
- * \brief Construct an IPAContextWrapper instance that wraps the \a context
- * \param[in] context The IPA module context
- *
- * Ownership of the \a context is passed to the IPAContextWrapper. The context remains
- * valid for the whole lifetime of the wrapper and is destroyed automatically
- * with it.
- */
-IPAContextWrapper::IPAContextWrapper(struct ipa_context *context)
-	: ctx_(context), intf_(nullptr)
-{
-	if (!ctx_)
-		return;
-
-	bool forceCApi = !!utils::secure_getenv("LIBCAMERA_IPA_FORCE_C_API");
-
-	if (!forceCApi && ctx_ && ctx_->ops->get_interface) {
-		intf_ = reinterpret_cast<IPAInterface *>(ctx_->ops->get_interface(ctx_));
-		intf_->queueFrameAction.connect(this, &IPAContextWrapper::doQueueFrameAction);
-		return;
-	}
-
-	ctx_->ops->register_callbacks(ctx_, &IPAContextWrapper::callbacks_,
-				      this);
-}
-
-IPAContextWrapper::~IPAContextWrapper()
-{
-	if (!ctx_)
-		return;
-
-	ctx_->ops->destroy(ctx_);
-}
-
-int IPAContextWrapper::init(const IPASettings &settings)
-{
-	if (intf_)
-		return intf_->init(settings);
-
-	if (!ctx_)
-		return 0;
-
-	struct ipa_settings c_settings;
-	c_settings.configuration_file = settings.configurationFile.c_str();
-
-	ctx_->ops->init(ctx_, &c_settings);
-
-	return 0;
-}
-
-int IPAContextWrapper::start(const IPAOperationData &data,
-			     IPAOperationData *result)
-{
-	if (intf_)
-		return intf_->start(data, result);
-
-	if (!ctx_)
-		return 0;
-
-	return ctx_->ops->start(ctx_);
-}
-
-void IPAContextWrapper::stop()
-{
-	if (intf_)
-		return intf_->stop();
-
-	if (!ctx_)
-		return;
-
-	ctx_->ops->stop(ctx_);
-}
-
-void IPAContextWrapper::configure(const CameraSensorInfo &sensorInfo,
-				  const std::map<unsigned int, IPAStream> &streamConfig,
-				  const std::map<unsigned int, const ControlInfoMap &> &entityControls,
-				  const IPAOperationData &ipaConfig,
-				  IPAOperationData *result)
-{
-	if (intf_)
-		return intf_->configure(sensorInfo, streamConfig,
-					entityControls, ipaConfig, result);
-
-	if (!ctx_)
-		return;
-
-	serializer_.reset();
-
-	/* Translate the camera sensor info. */
-	struct ipa_sensor_info sensor_info = {};
-	sensor_info.model = sensorInfo.model.c_str();
-	sensor_info.bits_per_pixel = sensorInfo.bitsPerPixel;
-	sensor_info.active_area.width = sensorInfo.activeAreaSize.width;
-	sensor_info.active_area.height = sensorInfo.activeAreaSize.height;
-	sensor_info.analog_crop.left = sensorInfo.analogCrop.x;
-	sensor_info.analog_crop.top = sensorInfo.analogCrop.y;
-	sensor_info.analog_crop.width = sensorInfo.analogCrop.width;
-	sensor_info.analog_crop.height = sensorInfo.analogCrop.height;
-	sensor_info.output_size.width = sensorInfo.outputSize.width;
-	sensor_info.output_size.height = sensorInfo.outputSize.height;
-	sensor_info.pixel_rate = sensorInfo.pixelRate;
-	sensor_info.line_length = sensorInfo.lineLength;
-
-	/* Translate the IPA stream configurations map. */
-	struct ipa_stream c_streams[streamConfig.size()];
-
-	unsigned int i = 0;
-	for (const auto &stream : streamConfig) {
-		struct ipa_stream *c_stream = &c_streams[i];
-		unsigned int id = stream.first;
-		const IPAStream &ipaStream = stream.second;
-
-		c_stream->id = id;
-		c_stream->pixel_format = ipaStream.pixelFormat;
-		c_stream->width = ipaStream.size.width;
-		c_stream->height = ipaStream.size.height;
-
-		++i;
-	}
-
-	/* Translate the IPA entity controls map. */
-	struct ipa_control_info_map c_info_maps[entityControls.size()];
-	std::vector<std::vector<uint8_t>> data(entityControls.size());
-
-	i = 0;
-	for (const auto &info : entityControls) {
-		struct ipa_control_info_map &c_info_map = c_info_maps[i];
-		unsigned int id = info.first;
-		const ControlInfoMap &infoMap = info.second;
-
-		size_t infoMapSize = serializer_.binarySize(infoMap);
-		data[i].resize(infoMapSize);
-		ByteStreamBuffer byteStream(data[i].data(), data[i].size());
-		serializer_.serialize(infoMap, byteStream);
-
-		c_info_map.id = id;
-		c_info_map.data = byteStream.base();
-		c_info_map.size = byteStream.size();
-
-		++i;
-	}
-
-	/* \todo Translate the ipaConfig and reponse */
-	ctx_->ops->configure(ctx_, &sensor_info, c_streams, streamConfig.size(),
-			     c_info_maps, entityControls.size());
-}
-
-void IPAContextWrapper::mapBuffers(const std::vector<IPABuffer> &buffers)
-{
-	if (intf_)
-		return intf_->mapBuffers(buffers);
-
-	if (!ctx_)
-		return;
-
-	struct ipa_buffer c_buffers[buffers.size()];
-
-	for (unsigned int i = 0; i < buffers.size(); ++i) {
-		struct ipa_buffer &c_buffer = c_buffers[i];
-		const IPABuffer &buffer = buffers[i];
-		const std::vector<FrameBuffer::Plane> &planes = buffer.planes;
-
-		c_buffer.id = buffer.id;
-		c_buffer.num_planes = planes.size();
-
-		for (unsigned int j = 0; j < planes.size(); ++j) {
-			const FrameBuffer::Plane &plane = planes[j];
-			c_buffer.planes[j].dmabuf = plane.fd.fd();
-			c_buffer.planes[j].length = plane.length;
-		}
-	}
-
-	ctx_->ops->map_buffers(ctx_, c_buffers, buffers.size());
-}
-
-void IPAContextWrapper::unmapBuffers(const std::vector<unsigned int> &ids)
-{
-	if (intf_)
-		return intf_->unmapBuffers(ids);
-
-	if (!ctx_)
-		return;
-
-	ctx_->ops->unmap_buffers(ctx_, ids.data(), ids.size());
-}
-
-void IPAContextWrapper::processEvent(const IPAOperationData &data)
-{
-	if (intf_)
-		return intf_->processEvent(data);
-
-	if (!ctx_)
-		return;
-
-	struct ipa_operation_data c_data;
-	c_data.operation = data.operation;
-	c_data.data = data.data.data();
-	c_data.num_data = data.data.size();
-
-	struct ipa_control_list control_lists[data.controls.size()];
-	c_data.lists = control_lists;
-	c_data.num_lists = data.controls.size();
-
-	std::size_t listsSize = 0;
-	for (const auto &list : data.controls)
-		listsSize += serializer_.binarySize(list);
-
-	std::vector<uint8_t> binaryData(listsSize);
-	ByteStreamBuffer byteStreamBuffer(binaryData.data(), listsSize);
-
-	unsigned int i = 0;
-	for (const auto &list : data.controls) {
-		struct ipa_control_list &c_list = control_lists[i];
-		c_list.size = serializer_.binarySize(list);
-		ByteStreamBuffer b = byteStreamBuffer.carveOut(c_list.size);
-
-		serializer_.serialize(list, b);
-
-		c_list.data = b.base();
-	}
-
-	ctx_->ops->process_event(ctx_, &c_data);
-}
-
-void IPAContextWrapper::doQueueFrameAction(unsigned int frame,
-					   const IPAOperationData &data)
-{
-	IPAInterface::queueFrameAction.emit(frame, data);
-}
-
-void IPAContextWrapper::queue_frame_action(void *ctx, unsigned int frame,
-					   struct ipa_operation_data &data)
-{
-	IPAContextWrapper *_this = static_cast<IPAContextWrapper *>(ctx);
-	IPAOperationData opData;
-
-	opData.operation = data.operation;
-	for (unsigned int i = 0; i < data.num_data; ++i)
-		opData.data.push_back(data.data[i]);
-
-	for (unsigned int i = 0; i < data.num_lists; ++i) {
-		const struct ipa_control_list &c_list = data.lists[i];
-		ByteStreamBuffer b(c_list.data, c_list.size);
-		opData.controls.push_back(_this->serializer_.deserialize<ControlList>(b));
-	}
-
-	_this->doQueueFrameAction(frame, opData);
-}
-
-#ifndef __DOXYGEN__
-/*
- * This construct confuses Doxygen and makes it believe that all members of the
- * operations is a member of IPAContextWrapper. It must thus be hidden.
- */
-const struct ipa_callback_ops IPAContextWrapper::callbacks_ = {
-	.queue_frame_action = &IPAContextWrapper::queue_frame_action,
-};
-#endif
-
-} /* namespace libcamera */