From e201cb4f5450a84fc669e313bc039174cf54c263 Mon Sep 17 00:00:00 2001 From: Paul Elder Date: Sat, 5 Dec 2020 19:30:54 +0900 Subject: libcamera: IPAInterface: Replace C API with the new C++-only API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove everything related to the C API, including ipa_context, ipa_context_wrapper, and IPAInterfaceWrapper. Also remove relevant documentation. ipaCreate() provided by IPA implementations, and createInterface() provided by IPAModule (wrapper around IPA implementation) both now return a C++ object IPAInterface instead of struct ipa_context. Although IPAInterfaceWrapper is the only component of libipa, the skeleton and build files for libipa are retained. After converting the C API to the C++-only API, make all pipeline handlers and IPAs use the new API. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- This is a combination of 21 commits: --- libcamera: IPAModule: Replace ipa_context with IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate_() and createInterface() return IPAInterface. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: ipa_context_wrapper: Remove ipa_context_wrapper Since ipa_context has been replaced with custom IPAInterfaces, it is not longer needed. Remove it. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAInterface: remove ipa_context and functions from documentation Remove all the documentation related to ipa_context and the C IPA API, as well as the documentation about the functions in the IPAInterface. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- libcamera: IPAInterface: Remove all functions from IPAInterface Now that all the functions in the IPA interface are defined in the data definition file and a specialized IPAInterface is generated per pipeline handler, remove all the functions from the base IPAInterface. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAInterface: make ipaCreate return IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate return IPAinterface. Signed-off-by: Paul Elder Reviewed-by: Jacopo Mondi --- ipa: remove IPAInterfaceWrapper As every pipeline has its own proxy, IPAInterfaceWrapper is no longer necessary. Remove it. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Acked-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- libcamera: IPAProxy: Remove stop() override Since stop() is part of the IPA interface, and the IPA interface is now generated based on the data definition file per pipeline, this no longer needs to be overrided by the base IPAProxy. Remove it. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAProxy, IPAManager: Switch to one-proxy-per-pipeline scheme IPAProxy is changed in two major ways: - Every pipeline has its own proxy, to support each pipeline's IPA interface - IPAProxy implementations always encapsulate IPA modules, and switch internally for isolation or threaded The IPAProxy registration mechanism is removed, as each pipeline will have its own proxy, so the pipeline can pass the specialized class name of the IPAProxy to the IPAManager for construction. IPAManager is changed accordingly to support these changes: - createIPA is a template function that takes an IPAProxy class, and always returns an IPAProxy - IPAManager no longer decides on isolation, and simply creates an IPAProxy instance while passing the isolation flag Consequently, the old IPAProxy classes (IPAProxyThread and IPAProxyLinux) are removed. The IPAInterfaceTest is updated to use the new IPAManager interface, and to construct a ProcessManager as no single global instance is created anymore. Signed-off-by: Paul Elder --- libcamera: IPAProxy: Add isolate parameter to create() Since IPAProxy implementations now always encapsulate IPA modules, add a parameter to create() to signal if the proxy should isolate the IPA or not. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline Now that each pipeline handler has its own IPAProxy implementation, make the IPAManager fetch the IPAProxy based on the pipeline handler name. Also, since the IPAProxy is used regardless of isolation or no isolation, remove the isolation check from the proxy selection. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- libcamera: IPAManager: add isolation flag to proxy creation When the IPA proxy is created, it needs to know whether to isolate or not. Feed the flag at creation of the IPA proxy. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAManager: Make createIPA return proxy directly Since every pipeline knows the type of the proxy that it needs, and since all IPAs are to be wrapped in a proxy, IPAManager no longer needs to search in the factory list to fetch the proxy factory to construct a factory. Instead, we define createIPA as a template function, and the pipeline handler can declare the proxy type when it calls createIPA. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAProxy: Remove registration mechanism Implementations of IPA proxies use a registration mechanism to register themselves with the main IPA proxy factory. This registration declares static objects, causing a risk of things being constructed before the proper libcamera facilities are ready. Since each pipeline handler has its own IPA proxy and knows the type, it isn't necessary to have a proxy factory. Remove it to alleviate the risk of early construction. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread We have now changed the proxy from per-IPC mechanism to per-pipeline. The per-IPC mechanism proxies are thus no longer needed; remove them. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- tests: ipa_interface_test: Update to use new createIPA Update the IPA interface test to use the new createIPA function from IPAManager. Also create an instance of ProcessManager, as no single global instance is created automatically anymore. Update meson.build to to depend on the generated IPA interface headers. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- libcamera: PipelineHandler: Remove IPA from base class Since pipeline handlers now have their own IPA interface types, it can no longer be defined in the base class, and each pipeline handler implementation must declare it and its type themselves. Remove it from the base class. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- ipa: raspberrypi: Add mojom data definition file Add a mojom data definition for raspberrypi pipeline handler's IPAs. This simplifies the API between the raspberrypi pipeline handler and the IPA, and is not a direct translation of what was used before with IPAOperationData. Also move the enums from raspberrypi.h to raspberrypi.mojom Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- libcamera: pipeline, ipa: raspberrypi: Use new data definition Now that we can generate custom functions and data structures with mojo, switch the raspberrypi pipeline handler and IPA to use the custom data structures as defined in the mojom data definition file. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- libcamera: pipeline, ipa: vimc: Support the new IPC mechanism Add support to vimc pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism Add support to the rkisp1 pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- libcamera: pipeline, ipa: ipu3: Support the new IPC mechanism Add support to ipu3 pipeline handler and IPA for the new IPC mechanism. [Original version] Signed-off-by: Niklas Söderlund Reviewed-by: Paul Elder [Fixed commit message and small changes] Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- src/ipa/libipa/ipa_interface_wrapper.cpp | 287 ------------------------------- src/ipa/libipa/ipa_interface_wrapper.h | 61 ------- src/ipa/libipa/meson.build | 2 - 3 files changed, 350 deletions(-) delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h (limited to 'src/ipa/libipa') diff --git a/src/ipa/libipa/ipa_interface_wrapper.cpp b/src/ipa/libipa/ipa_interface_wrapper.cpp deleted file mode 100644 index 40628489..00000000 --- a/src/ipa/libipa/ipa_interface_wrapper.cpp +++ /dev/null @@ -1,287 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_interface_wrapper.cpp - Image Processing Algorithm interface wrapper - */ - -#include "ipa_interface_wrapper.h" - -#include -#include -#include -#include - -#include - -#include "libcamera/internal/byte_stream_buffer.h" -#include "libcamera/internal/camera_sensor.h" - -/** - * \file ipa_interface_wrapper.h - * \brief Image Processing Algorithm interface wrapper - */ - -namespace libcamera { - -/** - * \class IPAInterfaceWrapper - * \brief Wrap an IPAInterface and expose it as an ipa_context - * - * This class implements the ipa_context API based on a provided IPAInterface. - * It helps IPAs that implement the IPAInterface API to provide the external - * ipa_context API. - * - * To use the wrapper, an IPA module simple creates a new instance of its - * IPAInterface implementation, and passes it to the constructor of the - * IPAInterfaceWrapper. As IPAInterfaceWrapper inherits from ipa_context, the - * constructed wrapper can then be directly returned from the IPA module's - * ipaCreate() function. - * - * \code{.cpp} - * class MyIPA : public IPAInterface - * { - * ... - * }; - * - * struct ipa_context *ipaCreate() - * { - * return new IPAInterfaceWrapper(std::make_unique()); - * } - * \endcode - * - * The wrapper takes ownership of the IPAInterface and will automatically - * delete it when the wrapper is destroyed. - */ - -/** - * \brief Construct an IPAInterfaceWrapper wrapping \a interface - * \param[in] interface The interface to wrap - */ -IPAInterfaceWrapper::IPAInterfaceWrapper(std::unique_ptr interface) - : ipa_(std::move(interface)), callbacks_(nullptr), cb_ctx_(nullptr) -{ - ops = &operations_; - - ipa_->queueFrameAction.connect(this, &IPAInterfaceWrapper::queueFrameAction); -} - -void IPAInterfaceWrapper::destroy(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - delete ctx; -} - -void *IPAInterfaceWrapper::get_interface(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - return ctx->ipa_.get(); -} - -void IPAInterfaceWrapper::init(struct ipa_context *_ctx, - const struct ipa_settings *settings) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - IPASettings ipaSettings{ - .configurationFile = settings->configuration_file - }; - ctx->ipa_->init(ipaSettings); -} - -int IPAInterfaceWrapper::start(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - /* \todo Translate the data and result. */ - IPAOperationData data = {}; - return ctx->ipa_->start(data, nullptr); -} - -void IPAInterfaceWrapper::stop(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - ctx->ipa_->stop(); -} - -void IPAInterfaceWrapper::register_callbacks(struct ipa_context *_ctx, - const struct ipa_callback_ops *callbacks, - void *cb_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - ctx->callbacks_ = callbacks; - ctx->cb_ctx_ = cb_ctx; -} - -void IPAInterfaceWrapper::configure(struct ipa_context *_ctx, - const struct ipa_sensor_info *sensor_info, - const struct ipa_stream *streams, - unsigned int num_streams, - const struct ipa_control_info_map *maps, - unsigned int num_maps) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - ctx->serializer_.reset(); - - /* Translate the IPA sensor info. */ - CameraSensorInfo sensorInfo{}; - sensorInfo.model = sensor_info->model; - sensorInfo.bitsPerPixel = sensor_info->bits_per_pixel; - sensorInfo.activeAreaSize = { sensor_info->active_area.width, - sensor_info->active_area.height }; - sensorInfo.analogCrop = { sensor_info->analog_crop.left, - sensor_info->analog_crop.top, - sensor_info->analog_crop.width, - sensor_info->analog_crop.height }; - sensorInfo.outputSize = { sensor_info->output_size.width, - sensor_info->output_size.height }; - sensorInfo.pixelRate = sensor_info->pixel_rate; - sensorInfo.lineLength = sensor_info->line_length; - - /* Translate the IPA stream configurations map. */ - std::map ipaStreams; - - for (unsigned int i = 0; i < num_streams; ++i) { - const struct ipa_stream &stream = streams[i]; - - ipaStreams[stream.id] = { - stream.pixel_format, - Size(stream.width, stream.height), - }; - } - - /* Translate the IPA entity controls map. */ - std::map entityControls; - std::map infoMaps; - - for (unsigned int i = 0; i < num_maps; ++i) { - const struct ipa_control_info_map &ipa_map = maps[i]; - ByteStreamBuffer byteStream(ipa_map.data, ipa_map.size); - unsigned int id = ipa_map.id; - - infoMaps[id] = ctx->serializer_.deserialize(byteStream); - entityControls.emplace(id, infoMaps[id]); - } - - /* \todo Translate the ipaConfig and result. */ - IPAOperationData ipaConfig; - ctx->ipa_->configure(sensorInfo, ipaStreams, entityControls, ipaConfig, - nullptr); -} - -void IPAInterfaceWrapper::map_buffers(struct ipa_context *_ctx, - const struct ipa_buffer *_buffers, - size_t num_buffers) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - std::vector buffers(num_buffers); - - for (unsigned int i = 0; i < num_buffers; ++i) { - const struct ipa_buffer &_buffer = _buffers[i]; - IPABuffer &buffer = buffers[i]; - std::vector &planes = buffer.planes; - - buffer.id = _buffer.id; - - planes.resize(_buffer.num_planes); - for (unsigned int j = 0; j < _buffer.num_planes; ++j) { - planes[j].fd = FileDescriptor(_buffer.planes[j].dmabuf); - planes[j].length = _buffer.planes[j].length; - } - } - - ctx->ipa_->mapBuffers(buffers); -} - -void IPAInterfaceWrapper::unmap_buffers(struct ipa_context *_ctx, - const unsigned int *_ids, - size_t num_buffers) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - std::vector ids(_ids, _ids + num_buffers); - ctx->ipa_->unmapBuffers(ids); -} - -void IPAInterfaceWrapper::process_event(struct ipa_context *_ctx, - const struct ipa_operation_data *data) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - IPAOperationData opData; - - opData.operation = data->operation; - - opData.data.resize(data->num_data); - memcpy(opData.data.data(), data->data, - data->num_data * sizeof(*data->data)); - - opData.controls.resize(data->num_lists); - for (unsigned int i = 0; i < data->num_lists; ++i) { - const struct ipa_control_list *c_list = &data->lists[i]; - ByteStreamBuffer byteStream(c_list->data, c_list->size); - opData.controls[i] = ctx->serializer_.deserialize(byteStream); - } - - ctx->ipa_->processEvent(opData); -} - -void IPAInterfaceWrapper::queueFrameAction(unsigned int frame, - const IPAOperationData &data) -{ - if (!callbacks_) - return; - - struct ipa_operation_data c_data; - c_data.operation = data.operation; - c_data.data = data.data.data(); - c_data.num_data = data.data.size(); - - struct ipa_control_list control_lists[data.controls.size()]; - c_data.lists = control_lists; - c_data.num_lists = data.controls.size(); - - std::size_t listsSize = 0; - for (const auto &list : data.controls) - listsSize += serializer_.binarySize(list); - - std::vector binaryData(listsSize); - ByteStreamBuffer byteStreamBuffer(binaryData.data(), listsSize); - - unsigned int i = 0; - for (const auto &list : data.controls) { - struct ipa_control_list &c_list = control_lists[i]; - c_list.size = serializer_.binarySize(list); - - ByteStreamBuffer b = byteStreamBuffer.carveOut(c_list.size); - serializer_.serialize(list, b); - - c_list.data = b.base(); - } - - callbacks_->queue_frame_action(cb_ctx_, frame, c_data); -} - -#ifndef __DOXYGEN__ -/* - * This construct confuses Doygen and makes it believe that all members of the - * operations is a member of IPAInterfaceWrapper. It must thus be hidden. - */ -const struct ipa_context_ops IPAInterfaceWrapper::operations_ = { - .destroy = &IPAInterfaceWrapper::destroy, - .get_interface = &IPAInterfaceWrapper::get_interface, - .init = &IPAInterfaceWrapper::init, - .start = &IPAInterfaceWrapper::start, - .stop = &IPAInterfaceWrapper::stop, - .register_callbacks = &IPAInterfaceWrapper::register_callbacks, - .configure = &IPAInterfaceWrapper::configure, - .map_buffers = &IPAInterfaceWrapper::map_buffers, - .unmap_buffers = &IPAInterfaceWrapper::unmap_buffers, - .process_event = &IPAInterfaceWrapper::process_event, -}; -#endif - -} /* namespace libcamera */ diff --git a/src/ipa/libipa/ipa_interface_wrapper.h b/src/ipa/libipa/ipa_interface_wrapper.h deleted file mode 100644 index a1c70159..00000000 --- a/src/ipa/libipa/ipa_interface_wrapper.h +++ /dev/null @@ -1,61 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_interface_wrapper.h - Image Processing Algorithm interface wrapper - */ -#ifndef __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ -#define __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ - -#include - -#include - -#include "libcamera/internal/control_serializer.h" - -namespace libcamera { - -class IPAInterfaceWrapper : public ipa_context -{ -public: - IPAInterfaceWrapper(std::unique_ptr interface); - -private: - static void destroy(struct ipa_context *ctx); - static void *get_interface(struct ipa_context *ctx); - static void init(struct ipa_context *ctx, - const struct ipa_settings *settings); - static int start(struct ipa_context *ctx); - static void stop(struct ipa_context *ctx); - static void register_callbacks(struct ipa_context *ctx, - const struct ipa_callback_ops *callbacks, - void *cb_ctx); - static void configure(struct ipa_context *ctx, - const struct ipa_sensor_info *sensor_info, - const struct ipa_stream *streams, - unsigned int num_streams, - const struct ipa_control_info_map *maps, - unsigned int num_maps); - static void map_buffers(struct ipa_context *ctx, - const struct ipa_buffer *c_buffers, - size_t num_buffers); - static void unmap_buffers(struct ipa_context *ctx, - const unsigned int *ids, - size_t num_buffers); - static void process_event(struct ipa_context *ctx, - const struct ipa_operation_data *data); - - static const struct ipa_context_ops operations_; - - void queueFrameAction(unsigned int frame, const IPAOperationData &data); - - std::unique_ptr ipa_; - const struct ipa_callback_ops *callbacks_; - void *cb_ctx_; - - ControlSerializer serializer_; -}; - -} /* namespace libcamera */ - -#endif /* __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 22626405..b29ef0f4 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,11 +1,9 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ - 'ipa_interface_wrapper.h', ]) libipa_sources = files([ - 'ipa_interface_wrapper.cpp', ]) libipa_includes = include_directories('..') -- cgit v1.2.1