From e201cb4f5450a84fc669e313bc039174cf54c263 Mon Sep 17 00:00:00 2001 From: Paul Elder Date: Sat, 5 Dec 2020 19:30:54 +0900 Subject: libcamera: IPAInterface: Replace C API with the new C++-only API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove everything related to the C API, including ipa_context, ipa_context_wrapper, and IPAInterfaceWrapper. Also remove relevant documentation. ipaCreate() provided by IPA implementations, and createInterface() provided by IPAModule (wrapper around IPA implementation) both now return a C++ object IPAInterface instead of struct ipa_context. Although IPAInterfaceWrapper is the only component of libipa, the skeleton and build files for libipa are retained. After converting the C API to the C++-only API, make all pipeline handlers and IPAs use the new API. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- This is a combination of 21 commits: --- libcamera: IPAModule: Replace ipa_context with IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate_() and createInterface() return IPAInterface. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: ipa_context_wrapper: Remove ipa_context_wrapper Since ipa_context has been replaced with custom IPAInterfaces, it is not longer needed. Remove it. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAInterface: remove ipa_context and functions from documentation Remove all the documentation related to ipa_context and the C IPA API, as well as the documentation about the functions in the IPAInterface. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- libcamera: IPAInterface: Remove all functions from IPAInterface Now that all the functions in the IPA interface are defined in the data definition file and a specialized IPAInterface is generated per pipeline handler, remove all the functions from the base IPAInterface. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAInterface: make ipaCreate return IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate return IPAinterface. Signed-off-by: Paul Elder Reviewed-by: Jacopo Mondi --- ipa: remove IPAInterfaceWrapper As every pipeline has its own proxy, IPAInterfaceWrapper is no longer necessary. Remove it. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Acked-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- libcamera: IPAProxy: Remove stop() override Since stop() is part of the IPA interface, and the IPA interface is now generated based on the data definition file per pipeline, this no longer needs to be overrided by the base IPAProxy. Remove it. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAProxy, IPAManager: Switch to one-proxy-per-pipeline scheme IPAProxy is changed in two major ways: - Every pipeline has its own proxy, to support each pipeline's IPA interface - IPAProxy implementations always encapsulate IPA modules, and switch internally for isolation or threaded The IPAProxy registration mechanism is removed, as each pipeline will have its own proxy, so the pipeline can pass the specialized class name of the IPAProxy to the IPAManager for construction. IPAManager is changed accordingly to support these changes: - createIPA is a template function that takes an IPAProxy class, and always returns an IPAProxy - IPAManager no longer decides on isolation, and simply creates an IPAProxy instance while passing the isolation flag Consequently, the old IPAProxy classes (IPAProxyThread and IPAProxyLinux) are removed. The IPAInterfaceTest is updated to use the new IPAManager interface, and to construct a ProcessManager as no single global instance is created anymore. Signed-off-by: Paul Elder --- libcamera: IPAProxy: Add isolate parameter to create() Since IPAProxy implementations now always encapsulate IPA modules, add a parameter to create() to signal if the proxy should isolate the IPA or not. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline Now that each pipeline handler has its own IPAProxy implementation, make the IPAManager fetch the IPAProxy based on the pipeline handler name. Also, since the IPAProxy is used regardless of isolation or no isolation, remove the isolation check from the proxy selection. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- libcamera: IPAManager: add isolation flag to proxy creation When the IPA proxy is created, it needs to know whether to isolate or not. Feed the flag at creation of the IPA proxy. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAManager: Make createIPA return proxy directly Since every pipeline knows the type of the proxy that it needs, and since all IPAs are to be wrapped in a proxy, IPAManager no longer needs to search in the factory list to fetch the proxy factory to construct a factory. Instead, we define createIPA as a template function, and the pipeline handler can declare the proxy type when it calls createIPA. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: IPAProxy: Remove registration mechanism Implementations of IPA proxies use a registration mechanism to register themselves with the main IPA proxy factory. This registration declares static objects, causing a risk of things being constructed before the proper libcamera facilities are ready. Since each pipeline handler has its own IPA proxy and knows the type, it isn't necessary to have a proxy factory. Remove it to alleviate the risk of early construction. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread We have now changed the proxy from per-IPC mechanism to per-pipeline. The per-IPC mechanism proxies are thus no longer needed; remove them. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- tests: ipa_interface_test: Update to use new createIPA Update the IPA interface test to use the new createIPA function from IPAManager. Also create an instance of ProcessManager, as no single global instance is created automatically anymore. Update meson.build to to depend on the generated IPA interface headers. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- libcamera: PipelineHandler: Remove IPA from base class Since pipeline handlers now have their own IPA interface types, it can no longer be defined in the base class, and each pipeline handler implementation must declare it and its type themselves. Remove it from the base class. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- ipa: raspberrypi: Add mojom data definition file Add a mojom data definition for raspberrypi pipeline handler's IPAs. This simplifies the API between the raspberrypi pipeline handler and the IPA, and is not a direct translation of what was used before with IPAOperationData. Also move the enums from raspberrypi.h to raspberrypi.mojom Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- libcamera: pipeline, ipa: raspberrypi: Use new data definition Now that we can generate custom functions and data structures with mojo, switch the raspberrypi pipeline handler and IPA to use the custom data structures as defined in the mojom data definition file. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- libcamera: pipeline, ipa: vimc: Support the new IPC mechanism Add support to vimc pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism Add support to the rkisp1 pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- libcamera: pipeline, ipa: ipu3: Support the new IPC mechanism Add support to ipu3 pipeline handler and IPA for the new IPC mechanism. [Original version] Signed-off-by: Niklas Söderlund Reviewed-by: Paul Elder [Fixed commit message and small changes] Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- src/ipa/ipu3/ipu3.cpp | 74 +-- src/ipa/ipu3/meson.build | 4 +- src/ipa/libipa/ipa_interface_wrapper.cpp | 287 --------- src/ipa/libipa/ipa_interface_wrapper.h | 61 -- src/ipa/libipa/meson.build | 2 - src/ipa/raspberrypi/raspberrypi.cpp | 180 +++--- src/ipa/rkisp1/meson.build | 2 +- src/ipa/rkisp1/rkisp1.cpp | 63 +- src/ipa/vimc/meson.build | 2 +- src/ipa/vimc/vimc.cpp | 39 +- src/libcamera/ipa_context_wrapper.cpp | 298 ---------- src/libcamera/ipa_interface.cpp | 644 ++------------------- src/libcamera/ipa_manager.cpp | 47 +- src/libcamera/ipa_module.cpp | 18 +- src/libcamera/ipa_proxy.cpp | 101 ---- src/libcamera/meson.build | 1 - src/libcamera/pipeline/ipu3/ipu3.cpp | 93 ++- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 257 ++++---- src/libcamera/pipeline/raspberrypi/rpi_stream.cpp | 6 +- src/libcamera/pipeline/raspberrypi/rpi_stream.h | 5 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 75 ++- src/libcamera/pipeline/vimc/vimc.cpp | 10 +- src/libcamera/pipeline_handler.cpp | 8 - src/libcamera/proxy/ipa_proxy_linux.cpp | 104 ---- src/libcamera/proxy/ipa_proxy_thread.cpp | 175 ------ src/libcamera/proxy/meson.build | 5 - .../proxy/worker/ipa_proxy_linux_worker.cpp | 90 --- src/libcamera/proxy/worker/meson.build | 4 - 28 files changed, 408 insertions(+), 2247 deletions(-) delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h delete mode 100644 src/libcamera/ipa_context_wrapper.cpp delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp (limited to 'src') diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b11b03ef..fcd8889c 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -5,8 +5,6 @@ * ipu3.cpp - IPU3 Image Processing Algorithms */ -#include - #include #include @@ -17,10 +15,9 @@ #include #include #include +#include #include -#include - #include "libcamera/internal/buffer.h" #include "libcamera/internal/log.h" @@ -28,25 +25,21 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPAIPU3) -class IPAIPU3 : public IPAInterface +class IPAIPU3 : public ipa::ipu3::IPAIPU3Interface { public: int init([[maybe_unused]] const IPASettings &settings) override { return 0; } - int start([[maybe_unused]] const IPAOperationData &data, - [[maybe_unused]] IPAOperationData *result) override { return 0; } + int start() override { return 0; } void stop() override {} - void configure(const CameraSensorInfo &info, - const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *response) override; + void configure(const std::map &entityControls) override; + void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPAOperationData &event) override; + void processEvent(const ipa::ipu3::IPU3Event &event) override; private: void processControls(unsigned int frame, const ControlList &controls); @@ -69,11 +62,7 @@ private: uint32_t maxGain_; }; -void IPAIPU3::configure([[maybe_unused]] const CameraSensorInfo &info, - [[maybe_unused]] const std::map &streamConfig, - const std::map &entityControls, - [[maybe_unused]] const IPAOperationData &ipaConfig, - [[maybe_unused]] IPAOperationData *result) +void IPAIPU3::configure(const std::map &entityControls) { if (entityControls.empty()) return; @@ -123,19 +112,15 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } } -void IPAIPU3::processEvent(const IPAOperationData &event) +void IPAIPU3::processEvent(const ipa::ipu3::IPU3Event &event) { - switch (event.operation) { - case IPU3_IPA_EVENT_PROCESS_CONTROLS: { - unsigned int frame = event.data[0]; - processControls(frame, event.controls[0]); + switch (event.op) { + case ipa::ipu3::EventProcessControls: { + processControls(event.frame, event.controls); break; } - case IPU3_IPA_EVENT_STAT_READY: { - unsigned int frame = event.data[0]; - unsigned int bufferId = event.data[1]; - - auto it = buffers_.find(bufferId); + case ipa::ipu3::EventStatReady: { + auto it = buffers_.find(event.bufferId); if (it == buffers_.end()) { LOG(IPAIPU3, Error) << "Could not find stats buffer!"; return; @@ -145,14 +130,11 @@ void IPAIPU3::processEvent(const IPAOperationData &event) const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - parseStatistics(frame, stats); + parseStatistics(event.frame, stats); break; } - case IPU3_IPA_EVENT_FILL_PARAMS: { - unsigned int frame = event.data[0]; - unsigned int bufferId = event.data[1]; - - auto it = buffers_.find(bufferId); + case ipa::ipu3::EventFillParams: { + auto it = buffers_.find(event.bufferId); if (it == buffers_.end()) { LOG(IPAIPU3, Error) << "Could not find param buffer!"; return; @@ -162,11 +144,11 @@ void IPAIPU3::processEvent(const IPAOperationData &event) ipu3_uapi_params *params = reinterpret_cast(mem.data()); - fillParams(frame, params); + fillParams(event.frame, params); break; } default: - LOG(IPAIPU3, Error) << "Unknown event " << event.operation; + LOG(IPAIPU3, Error) << "Unknown event " << event.op; break; } } @@ -184,8 +166,8 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) /* \todo Fill in parameters buffer. */ - IPAOperationData op; - op.operation = IPU3_IPA_ACTION_PARAM_FILLED; + ipa::ipu3::IPU3Action op; + op.op = ipa::ipu3::ActionParamFilled; queueFrameAction.emit(frame, op); @@ -201,22 +183,22 @@ void IPAIPU3::parseStatistics(unsigned int frame, /* \todo React to statistics and update internal state machine. */ /* \todo Add meta-data information to ctrls. */ - IPAOperationData op; - op.operation = IPU3_IPA_ACTION_METADATA_READY; - op.controls.push_back(ctrls); + ipa::ipu3::IPU3Action op; + op.op = ipa::ipu3::ActionMetadataReady; + op.controls = ctrls; queueFrameAction.emit(frame, op); } void IPAIPU3::setControls(unsigned int frame) { - IPAOperationData op; - op.operation = IPU3_IPA_ACTION_SET_SENSOR_CONTROLS; + ipa::ipu3::IPU3Action op; + op.op = ipa::ipu3::ActionSetSensorControls; ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls.push_back(ctrls); + op.controls = ctrls; queueFrameAction.emit(frame, op); } @@ -233,9 +215,9 @@ const struct IPAModuleInfo ipaModuleInfo = { "ipu3", }; -struct ipa_context *ipaCreate() +IPAInterface *ipaCreate() { - return new IPAInterfaceWrapper(std::make_unique()); + return new IPAIPU3(); } } diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index d31594fc..a241f617 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -3,10 +3,10 @@ ipa_name = 'ipa_ipu3' mod = shared_module(ipa_name, - 'ipu3.cpp', + ['ipu3.cpp', libcamera_generated_ipa_headers], name_prefix : '', include_directories : [ipa_includes, libipa_includes], - dependencies : [libatomic, libcamera_dep], + dependencies : libcamera_dep, link_with : libipa, install : true, install_dir : ipa_install_dir) diff --git a/src/ipa/libipa/ipa_interface_wrapper.cpp b/src/ipa/libipa/ipa_interface_wrapper.cpp deleted file mode 100644 index 40628489..00000000 --- a/src/ipa/libipa/ipa_interface_wrapper.cpp +++ /dev/null @@ -1,287 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_interface_wrapper.cpp - Image Processing Algorithm interface wrapper - */ - -#include "ipa_interface_wrapper.h" - -#include -#include -#include -#include - -#include - -#include "libcamera/internal/byte_stream_buffer.h" -#include "libcamera/internal/camera_sensor.h" - -/** - * \file ipa_interface_wrapper.h - * \brief Image Processing Algorithm interface wrapper - */ - -namespace libcamera { - -/** - * \class IPAInterfaceWrapper - * \brief Wrap an IPAInterface and expose it as an ipa_context - * - * This class implements the ipa_context API based on a provided IPAInterface. - * It helps IPAs that implement the IPAInterface API to provide the external - * ipa_context API. - * - * To use the wrapper, an IPA module simple creates a new instance of its - * IPAInterface implementation, and passes it to the constructor of the - * IPAInterfaceWrapper. As IPAInterfaceWrapper inherits from ipa_context, the - * constructed wrapper can then be directly returned from the IPA module's - * ipaCreate() function. - * - * \code{.cpp} - * class MyIPA : public IPAInterface - * { - * ... - * }; - * - * struct ipa_context *ipaCreate() - * { - * return new IPAInterfaceWrapper(std::make_unique()); - * } - * \endcode - * - * The wrapper takes ownership of the IPAInterface and will automatically - * delete it when the wrapper is destroyed. - */ - -/** - * \brief Construct an IPAInterfaceWrapper wrapping \a interface - * \param[in] interface The interface to wrap - */ -IPAInterfaceWrapper::IPAInterfaceWrapper(std::unique_ptr interface) - : ipa_(std::move(interface)), callbacks_(nullptr), cb_ctx_(nullptr) -{ - ops = &operations_; - - ipa_->queueFrameAction.connect(this, &IPAInterfaceWrapper::queueFrameAction); -} - -void IPAInterfaceWrapper::destroy(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - delete ctx; -} - -void *IPAInterfaceWrapper::get_interface(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - return ctx->ipa_.get(); -} - -void IPAInterfaceWrapper::init(struct ipa_context *_ctx, - const struct ipa_settings *settings) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - IPASettings ipaSettings{ - .configurationFile = settings->configuration_file - }; - ctx->ipa_->init(ipaSettings); -} - -int IPAInterfaceWrapper::start(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - /* \todo Translate the data and result. */ - IPAOperationData data = {}; - return ctx->ipa_->start(data, nullptr); -} - -void IPAInterfaceWrapper::stop(struct ipa_context *_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - ctx->ipa_->stop(); -} - -void IPAInterfaceWrapper::register_callbacks(struct ipa_context *_ctx, - const struct ipa_callback_ops *callbacks, - void *cb_ctx) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - ctx->callbacks_ = callbacks; - ctx->cb_ctx_ = cb_ctx; -} - -void IPAInterfaceWrapper::configure(struct ipa_context *_ctx, - const struct ipa_sensor_info *sensor_info, - const struct ipa_stream *streams, - unsigned int num_streams, - const struct ipa_control_info_map *maps, - unsigned int num_maps) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - - ctx->serializer_.reset(); - - /* Translate the IPA sensor info. */ - CameraSensorInfo sensorInfo{}; - sensorInfo.model = sensor_info->model; - sensorInfo.bitsPerPixel = sensor_info->bits_per_pixel; - sensorInfo.activeAreaSize = { sensor_info->active_area.width, - sensor_info->active_area.height }; - sensorInfo.analogCrop = { sensor_info->analog_crop.left, - sensor_info->analog_crop.top, - sensor_info->analog_crop.width, - sensor_info->analog_crop.height }; - sensorInfo.outputSize = { sensor_info->output_size.width, - sensor_info->output_size.height }; - sensorInfo.pixelRate = sensor_info->pixel_rate; - sensorInfo.lineLength = sensor_info->line_length; - - /* Translate the IPA stream configurations map. */ - std::map ipaStreams; - - for (unsigned int i = 0; i < num_streams; ++i) { - const struct ipa_stream &stream = streams[i]; - - ipaStreams[stream.id] = { - stream.pixel_format, - Size(stream.width, stream.height), - }; - } - - /* Translate the IPA entity controls map. */ - std::map entityControls; - std::map infoMaps; - - for (unsigned int i = 0; i < num_maps; ++i) { - const struct ipa_control_info_map &ipa_map = maps[i]; - ByteStreamBuffer byteStream(ipa_map.data, ipa_map.size); - unsigned int id = ipa_map.id; - - infoMaps[id] = ctx->serializer_.deserialize(byteStream); - entityControls.emplace(id, infoMaps[id]); - } - - /* \todo Translate the ipaConfig and result. */ - IPAOperationData ipaConfig; - ctx->ipa_->configure(sensorInfo, ipaStreams, entityControls, ipaConfig, - nullptr); -} - -void IPAInterfaceWrapper::map_buffers(struct ipa_context *_ctx, - const struct ipa_buffer *_buffers, - size_t num_buffers) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - std::vector buffers(num_buffers); - - for (unsigned int i = 0; i < num_buffers; ++i) { - const struct ipa_buffer &_buffer = _buffers[i]; - IPABuffer &buffer = buffers[i]; - std::vector &planes = buffer.planes; - - buffer.id = _buffer.id; - - planes.resize(_buffer.num_planes); - for (unsigned int j = 0; j < _buffer.num_planes; ++j) { - planes[j].fd = FileDescriptor(_buffer.planes[j].dmabuf); - planes[j].length = _buffer.planes[j].length; - } - } - - ctx->ipa_->mapBuffers(buffers); -} - -void IPAInterfaceWrapper::unmap_buffers(struct ipa_context *_ctx, - const unsigned int *_ids, - size_t num_buffers) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - std::vector ids(_ids, _ids + num_buffers); - ctx->ipa_->unmapBuffers(ids); -} - -void IPAInterfaceWrapper::process_event(struct ipa_context *_ctx, - const struct ipa_operation_data *data) -{ - IPAInterfaceWrapper *ctx = static_cast(_ctx); - IPAOperationData opData; - - opData.operation = data->operation; - - opData.data.resize(data->num_data); - memcpy(opData.data.data(), data->data, - data->num_data * sizeof(*data->data)); - - opData.controls.resize(data->num_lists); - for (unsigned int i = 0; i < data->num_lists; ++i) { - const struct ipa_control_list *c_list = &data->lists[i]; - ByteStreamBuffer byteStream(c_list->data, c_list->size); - opData.controls[i] = ctx->serializer_.deserialize(byteStream); - } - - ctx->ipa_->processEvent(opData); -} - -void IPAInterfaceWrapper::queueFrameAction(unsigned int frame, - const IPAOperationData &data) -{ - if (!callbacks_) - return; - - struct ipa_operation_data c_data; - c_data.operation = data.operation; - c_data.data = data.data.data(); - c_data.num_data = data.data.size(); - - struct ipa_control_list control_lists[data.controls.size()]; - c_data.lists = control_lists; - c_data.num_lists = data.controls.size(); - - std::size_t listsSize = 0; - for (const auto &list : data.controls) - listsSize += serializer_.binarySize(list); - - std::vector binaryData(listsSize); - ByteStreamBuffer byteStreamBuffer(binaryData.data(), listsSize); - - unsigned int i = 0; - for (const auto &list : data.controls) { - struct ipa_control_list &c_list = control_lists[i]; - c_list.size = serializer_.binarySize(list); - - ByteStreamBuffer b = byteStreamBuffer.carveOut(c_list.size); - serializer_.serialize(list, b); - - c_list.data = b.base(); - } - - callbacks_->queue_frame_action(cb_ctx_, frame, c_data); -} - -#ifndef __DOXYGEN__ -/* - * This construct confuses Doygen and makes it believe that all members of the - * operations is a member of IPAInterfaceWrapper. It must thus be hidden. - */ -const struct ipa_context_ops IPAInterfaceWrapper::operations_ = { - .destroy = &IPAInterfaceWrapper::destroy, - .get_interface = &IPAInterfaceWrapper::get_interface, - .init = &IPAInterfaceWrapper::init, - .start = &IPAInterfaceWrapper::start, - .stop = &IPAInterfaceWrapper::stop, - .register_callbacks = &IPAInterfaceWrapper::register_callbacks, - .configure = &IPAInterfaceWrapper::configure, - .map_buffers = &IPAInterfaceWrapper::map_buffers, - .unmap_buffers = &IPAInterfaceWrapper::unmap_buffers, - .process_event = &IPAInterfaceWrapper::process_event, -}; -#endif - -} /* namespace libcamera */ diff --git a/src/ipa/libipa/ipa_interface_wrapper.h b/src/ipa/libipa/ipa_interface_wrapper.h deleted file mode 100644 index a1c70159..00000000 --- a/src/ipa/libipa/ipa_interface_wrapper.h +++ /dev/null @@ -1,61 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_interface_wrapper.h - Image Processing Algorithm interface wrapper - */ -#ifndef __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ -#define __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ - -#include - -#include - -#include "libcamera/internal/control_serializer.h" - -namespace libcamera { - -class IPAInterfaceWrapper : public ipa_context -{ -public: - IPAInterfaceWrapper(std::unique_ptr interface); - -private: - static void destroy(struct ipa_context *ctx); - static void *get_interface(struct ipa_context *ctx); - static void init(struct ipa_context *ctx, - const struct ipa_settings *settings); - static int start(struct ipa_context *ctx); - static void stop(struct ipa_context *ctx); - static void register_callbacks(struct ipa_context *ctx, - const struct ipa_callback_ops *callbacks, - void *cb_ctx); - static void configure(struct ipa_context *ctx, - const struct ipa_sensor_info *sensor_info, - const struct ipa_stream *streams, - unsigned int num_streams, - const struct ipa_control_info_map *maps, - unsigned int num_maps); - static void map_buffers(struct ipa_context *ctx, - const struct ipa_buffer *c_buffers, - size_t num_buffers); - static void unmap_buffers(struct ipa_context *ctx, - const unsigned int *ids, - size_t num_buffers); - static void process_event(struct ipa_context *ctx, - const struct ipa_operation_data *data); - - static const struct ipa_context_ops operations_; - - void queueFrameAction(unsigned int frame, const IPAOperationData &data); - - std::unique_ptr ipa_; - const struct ipa_callback_ops *callbacks_; - void *cb_ctx_; - - ControlSerializer serializer_; -}; - -} /* namespace libcamera */ - -#endif /* __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 22626405..b29ef0f4 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,11 +1,9 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ - 'ipa_interface_wrapper.h', ]) libipa_sources = files([ - 'ipa_interface_wrapper.cpp', ]) libipa_includes = include_directories('..') diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index ef8b19ba..81a3195c 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -20,11 +20,10 @@ #include #include #include +#include #include #include -#include - #include "libcamera/internal/buffer.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/log.h" @@ -64,7 +63,7 @@ constexpr double defaultMaxFrameDuration = 1e6 / 0.01; LOG_DEFINE_CATEGORY(IPARPI) -class IPARPi : public IPAInterface +class IPARPi : public ipa::rpi::IPARPiInterface { public: IPARPi() @@ -77,21 +76,24 @@ public: ~IPARPi() { if (lsTable_) - munmap(lsTable_, RPi::MaxLsGridSize); + munmap(lsTable_, ipa::rpi::MaxLsGridSize); } int init(const IPASettings &settings) override; - int start(const IPAOperationData &data, IPAOperationData *result) override; + void start(const ipa::rpi::StartControls &data, + ipa::rpi::StartControls *result) override; void stop() override {} void configure(const CameraSensorInfo &sensorInfo, const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *response) override; + const std::map &entityControls, + const ipa::rpi::ConfigInput &data, + ipa::rpi::ConfigOutput *response, int32_t *ret) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPAOperationData &event) override; + void signalStatReady(const uint32_t bufferId) override; + void signalQueueRequest(const ControlList &controls) override; + void signalIspPrepare(const ipa::rpi::ISPConfig &data) override; private: void setMode(const CameraSensorInfo &sensorInfo); @@ -166,15 +168,15 @@ int IPARPi::init(const IPASettings &settings) return 0; } -int IPARPi::start(const IPAOperationData &data, IPAOperationData *result) +void IPARPi::start(const ipa::rpi::StartControls &data, + ipa::rpi::StartControls *result) { RPiController::Metadata metadata; ASSERT(result); - result->operation = 0; - if (data.operation & RPi::IPA_CONFIG_STARTUP_CTRLS) { + if (!data.controls.empty()) { /* We have been given some controls to action before start. */ - queueRequest(data.controls[0]); + queueRequest(data.controls); } controller_.SwitchMode(mode_, &metadata); @@ -189,8 +191,7 @@ int IPARPi::start(const IPAOperationData &data, IPAOperationData *result) if (agcStatus.shutter_time != 0.0 && agcStatus.analogue_gain != 0.0) { ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - result->controls.emplace_back(ctrls); - result->operation |= RPi::IPA_RESULT_SENSOR_CTRLS; + result->controls = std::move(ctrls); } /* @@ -237,12 +238,9 @@ int IPARPi::start(const IPAOperationData &data, IPAOperationData *result) mistrustCount_ = helper_->MistrustFramesModeSwitch(); } - result->data.push_back(dropFrame); - result->operation |= RPi::IPA_RESULT_DROP_FRAMES; + result->dropFrameCount = dropFrame; firstStart_ = false; - - return 0; } void IPARPi::setMode(const CameraSensorInfo &sensorInfo) @@ -292,30 +290,30 @@ void IPARPi::setMode(const CameraSensorInfo &sensorInfo) void IPARPi::configure(const CameraSensorInfo &sensorInfo, [[maybe_unused]] const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *result) + const std::map &entityControls, + const ipa::rpi::ConfigInput &ipaConfig, + ipa::rpi::ConfigOutput *result, int32_t *ret) { if (entityControls.size() != 2) { LOG(IPARPI, Error) << "No ISP or sensor controls found."; - result->operation = RPi::IPA_RESULT_CONFIG_FAILED; + *ret = -1; return; } - result->operation = 0; + result->params = 0; sensorCtrls_ = entityControls.at(0); ispCtrls_ = entityControls.at(1); if (!validateSensorControls()) { LOG(IPARPI, Error) << "Sensor control validation failed."; - result->operation = RPi::IPA_RESULT_CONFIG_FAILED; + *ret = -1; return; } if (!validateIspControls()) { LOG(IPARPI, Error) << "ISP control validation failed."; - result->operation = RPi::IPA_RESULT_CONFIG_FAILED; + *ret = -1; return; } @@ -334,7 +332,7 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, if (!helper_) { LOG(IPARPI, Error) << "Could not create camera helper for " << cameraName; - result->operation = RPi::IPA_RESULT_CONFIG_FAILED; + *ret = -1; return; } @@ -346,35 +344,30 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, helper_->GetDelays(exposureDelay, gainDelay); sensorMetadata = helper_->SensorEmbeddedDataPresent(); - result->data.push_back(gainDelay); - result->data.push_back(exposureDelay); /* For EXPOSURE ctrl */ - result->data.push_back(exposureDelay); /* For VBLANK ctrl */ - result->data.push_back(sensorMetadata); - - result->operation |= RPi::IPA_RESULT_SENSOR_PARAMS; + result->params |= ipa::rpi::ConfigStaggeredWrite; + result->sensorConfig.gainDelay = gainDelay; + result->sensorConfig.exposureDelay = exposureDelay; + result->sensorConfig.vblank = exposureDelay; + result->sensorConfig.sensorMetadata = sensorMetadata; } /* Re-assemble camera mode using the sensor info. */ setMode(sensorInfo); - /* - * The ipaConfig.data always gives us the user transform first. Note that - * this will always make the LS table pointer (if present) element 1. - */ - mode_.transform = static_cast(ipaConfig.data[0]); + mode_.transform = static_cast(ipaConfig.transform); /* Store the lens shading table pointer and handle if available. */ - if (ipaConfig.operation & RPi::IPA_CONFIG_LS_TABLE) { + if (ipaConfig.lsTableHandle.isValid()) { /* Remove any previous table, if there was one. */ if (lsTable_) { - munmap(lsTable_, RPi::MaxLsGridSize); + munmap(lsTable_, ipa::rpi::MaxLsGridSize); lsTable_ = nullptr; } - /* Map the LS table buffer into user space (now element 1). */ - lsTableHandle_ = FileDescriptor(ipaConfig.data[1]); + /* Map the LS table buffer into user space. */ + lsTableHandle_ = std::move(ipaConfig.lsTableHandle); if (lsTableHandle_.isValid()) { - lsTable_ = mmap(nullptr, RPi::MaxLsGridSize, PROT_READ | PROT_WRITE, + lsTable_ = mmap(nullptr, ipa::rpi::MaxLsGridSize, PROT_READ | PROT_WRITE, MAP_SHARED, lsTableHandle_.fd(), 0); if (lsTable_ == MAP_FAILED) { @@ -403,11 +396,12 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, agcStatus.analogue_gain = DefaultAnalogueGain; applyAGC(&agcStatus, ctrls); - result->controls.emplace_back(ctrls); - result->operation |= RPi::IPA_RESULT_SENSOR_CTRLS; + result->controls = std::move(ctrls); } lastMode_ = mode_; + + *ret = 0; } void IPARPi::mapBuffers(const std::vector &buffers) @@ -429,56 +423,35 @@ void IPARPi::unmapBuffers(const std::vector &ids) } } -void IPARPi::processEvent(const IPAOperationData &event) +void IPARPi::signalStatReady(uint32_t bufferId) { - switch (event.operation) { - case RPi::IPA_EVENT_SIGNAL_STAT_READY: { - unsigned int bufferId = event.data[0]; - - if (++checkCount_ != frameCount_) /* assert here? */ - LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; - if (frameCount_ > mistrustCount_) - processStats(bufferId); - - reportMetadata(); - - IPAOperationData op; - op.operation = RPi::IPA_ACTION_STATS_METADATA_COMPLETE; - op.data = { bufferId & RPi::BufferMask::ID }; - op.controls = { libcameraMetadata_ }; - queueFrameAction.emit(0, op); - break; - } + if (++checkCount_ != frameCount_) /* assert here? */ + LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; + if (frameCount_ > mistrustCount_) + processStats(bufferId); - case RPi::IPA_EVENT_SIGNAL_ISP_PREPARE: { - unsigned int embeddedbufferId = event.data[0]; - unsigned int bayerbufferId = event.data[1]; + reportMetadata(); - /* - * At start-up, or after a mode-switch, we may want to - * avoid running the control algos for a few frames in case - * they are "unreliable". - */ - prepareISP(embeddedbufferId); - frameCount_++; - - /* Ready to push the input buffer into the ISP. */ - IPAOperationData op; - op.operation = RPi::IPA_ACTION_RUN_ISP; - op.data = { bayerbufferId & RPi::BufferMask::ID }; - queueFrameAction.emit(0, op); - break; - } + statsMetadataComplete.emit(bufferId & ipa::rpi::MaskID, libcameraMetadata_); +} - case RPi::IPA_EVENT_QUEUE_REQUEST: { - queueRequest(event.controls[0]); - break; - } +void IPARPi::signalQueueRequest(const ControlList &controls) +{ + queueRequest(controls); +} - default: - LOG(IPARPI, Error) << "Unknown event " << event.operation; - break; - } +void IPARPi::signalIspPrepare(const ipa::rpi::ISPConfig &data) +{ + /* + * At start-up, or after a mode-switch, we may want to + * avoid running the control algos for a few frames in case + * they are "unreliable". + */ + prepareISP(data.embeddedbufferId); + frameCount_++; + + /* Ready to push the input buffer into the ISP. */ + runIsp.emit(data.bayerbufferId & ipa::rpi::MaskID); } void IPARPi::reportMetadata() @@ -933,10 +906,7 @@ void IPARPi::queueRequest(const ControlList &controls) void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) { - IPAOperationData op; - op.operation = RPi::IPA_ACTION_EMBEDDED_COMPLETE; - op.data = { bufferId & RPi::BufferMask::ID }; - queueFrameAction.emit(0, op); + embeddedComplete.emit(bufferId & ipa::rpi::MaskID); } void IPARPi::prepareISP(unsigned int bufferId) @@ -997,12 +967,8 @@ void IPARPi::prepareISP(unsigned int bufferId) if (dpcStatus) applyDPC(dpcStatus, ctrls); - if (!ctrls.empty()) { - IPAOperationData op; - op.operation = RPi::IPA_ACTION_V4L2_SET_ISP; - op.controls.push_back(ctrls); - queueFrameAction.emit(0, op); - } + if (!ctrls.empty()) + setIsp.emit(ctrls); } } @@ -1059,10 +1025,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - IPAOperationData op; - op.operation = RPi::IPA_ACTION_SET_DELAYED_CTRLS; - op.controls.emplace_back(ctrls); - queueFrameAction.emit(0, op); + setDelayedControls.emit(ctrls); } } @@ -1301,13 +1264,14 @@ void IPARPi::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls) .grid_width = w, .grid_stride = w, .grid_height = h, - .dmabuf = lsTableHandle_.fd(), + /* .dmabuf will be filled in by pipeline handler. */ + .dmabuf = 0, .ref_transform = 0, .corner_sampled = 1, .gain_format = GAIN_FORMAT_U4P10 }; - if (!lsTable_ || w * h * 4 * sizeof(uint16_t) > RPi::MaxLsGridSize) { + if (!lsTable_ || w * h * 4 * sizeof(uint16_t) > ipa::rpi::MaxLsGridSize) { LOG(IPARPI, Error) << "Do not have a correctly allocate lens shading table!"; return; } @@ -1378,9 +1342,9 @@ const struct IPAModuleInfo ipaModuleInfo = { "raspberrypi", }; -struct ipa_context *ipaCreate() +IPAInterface *ipaCreate() { - return new IPAInterfaceWrapper(std::make_unique()); + return new IPARPi(); } } /* extern "C" */ diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index 95eb5393..1a1c7159 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -3,7 +3,7 @@ ipa_name = 'ipa_rkisp1' mod = shared_module(ipa_name, - 'rkisp1.cpp', + ['rkisp1.cpp', libcamera_generated_ipa_headers], name_prefix : '', include_directories : [ipa_includes, libipa_includes], dependencies : libcamera_dep, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 39783abd..67bac986 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -19,36 +19,31 @@ #include #include #include -#include +#include #include -#include - #include "libcamera/internal/log.h" namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) -class IPARkISP1 : public IPAInterface +class IPARkISP1 : public ipa::rkisp1::IPARkISP1Interface { public: int init([[maybe_unused]] const IPASettings &settings) override { return 0; } - int start([[maybe_unused]] const IPAOperationData &data, - [[maybe_unused]] IPAOperationData *result) override { return 0; } + int start() override { return 0; } void stop() override {} void configure(const CameraSensorInfo &info, - const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *response) override; + const std::map &streamConfig, + const std::map &entityControls) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPAOperationData &event) override; + void processEvent(const ipa::rkisp1::RkISP1Event &event) override; private: void queueRequest(unsigned int frame, rkisp1_params_cfg *params, @@ -81,10 +76,8 @@ private: * before accessing them. */ void IPARkISP1::configure([[maybe_unused]] const CameraSensorInfo &info, - [[maybe_unused]] const std::map &streamConfig, - const std::map &entityControls, - [[maybe_unused]] const IPAOperationData &ipaConfig, - [[maybe_unused]] IPAOperationData *result) + [[maybe_unused]] const std::map &streamConfig, + const std::map &entityControls) { if (entityControls.empty()) return; @@ -160,12 +153,12 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) } } -void IPARkISP1::processEvent(const IPAOperationData &event) +void IPARkISP1::processEvent(const ipa::rkisp1::RkISP1Event &event) { - switch (event.operation) { - case RKISP1_IPA_EVENT_SIGNAL_STAT_BUFFER: { - unsigned int frame = event.data[0]; - unsigned int bufferId = event.data[1]; + switch (event.op) { + case ipa::rkisp1::EventSignalStatBuffer: { + unsigned int frame = event.frame; + unsigned int bufferId = event.bufferId; const rkisp1_stat_buffer *stats = static_cast(buffersMemory_[bufferId]); @@ -173,18 +166,18 @@ void IPARkISP1::processEvent(const IPAOperationData &event) updateStatistics(frame, stats); break; } - case RKISP1_IPA_EVENT_QUEUE_REQUEST: { - unsigned int frame = event.data[0]; - unsigned int bufferId = event.data[1]; + case ipa::rkisp1::EventQueueRequest: { + unsigned int frame = event.frame; + unsigned int bufferId = event.bufferId; rkisp1_params_cfg *params = static_cast(buffersMemory_[bufferId]); - queueRequest(frame, params, event.controls[0]); + queueRequest(frame, params, event.controls); break; } default: - LOG(IPARkISP1, Error) << "Unknown event " << event.operation; + LOG(IPARkISP1, Error) << "Unknown event " << event.op; break; } } @@ -204,8 +197,8 @@ void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, params->module_en_update = RKISP1_CIF_ISP_MODULE_AEC; } - IPAOperationData op; - op.operation = RKISP1_IPA_ACTION_PARAM_FILLED; + ipa::rkisp1::RkISP1Action op; + op.op = ipa::rkisp1::ActionParamFilled; queueFrameAction.emit(frame, op); } @@ -257,13 +250,13 @@ void IPARkISP1::updateStatistics(unsigned int frame, void IPARkISP1::setControls(unsigned int frame) { - IPAOperationData op; - op.operation = RKISP1_IPA_ACTION_V4L2_SET; + ipa::rkisp1::RkISP1Action op; + op.op = ipa::rkisp1::ActionV4L2Set; ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls.push_back(ctrls); + op.controls = ctrls; queueFrameAction.emit(frame, op); } @@ -275,9 +268,9 @@ void IPARkISP1::metadataReady(unsigned int frame, unsigned int aeState) if (aeState) ctrls.set(controls::AeLocked, aeState == 2); - IPAOperationData op; - op.operation = RKISP1_IPA_ACTION_METADATA; - op.controls.push_back(ctrls); + ipa::rkisp1::RkISP1Action op; + op.op = ipa::rkisp1::ActionMetadata; + op.controls = ctrls; queueFrameAction.emit(frame, op); } @@ -294,9 +287,9 @@ const struct IPAModuleInfo ipaModuleInfo = { "rkisp1", }; -struct ipa_context *ipaCreate() +IPAInterface *ipaCreate() { - return new IPAInterfaceWrapper(std::make_unique()); + return new IPARkISP1(); } } diff --git a/src/ipa/vimc/meson.build b/src/ipa/vimc/meson.build index b1544c28..a35825ae 100644 --- a/src/ipa/vimc/meson.build +++ b/src/ipa/vimc/meson.build @@ -3,7 +3,7 @@ ipa_name = 'ipa_vimc' mod = shared_module(ipa_name, - 'vimc.cpp', + ['vimc.cpp', libcamera_generated_ipa_headers], name_prefix : '', include_directories : [ipa_includes, libipa_includes], dependencies : libcamera_dep, diff --git a/src/ipa/vimc/vimc.cpp b/src/ipa/vimc/vimc.cpp index 074902ee..13681d88 100644 --- a/src/ipa/vimc/vimc.cpp +++ b/src/ipa/vimc/vimc.cpp @@ -5,7 +5,7 @@ * ipa_vimc.cpp - Vimc Image Processing Algorithm module */ -#include +#include #include #include @@ -17,8 +17,6 @@ #include #include -#include - #include "libcamera/internal/file.h" #include "libcamera/internal/log.h" @@ -26,7 +24,7 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPAVimc) -class IPAVimc : public IPAInterface +class IPAVimc : public ipa::vimc::IPAVimcInterface { public: IPAVimc(); @@ -34,22 +32,12 @@ public: int init(const IPASettings &settings) override; - int start(const IPAOperationData &data, - IPAOperationData *result) override; + int start() override; void stop() override; - void configure([[maybe_unused]] const CameraSensorInfo &sensorInfo, - [[maybe_unused]] const std::map &streamConfig, - [[maybe_unused]] const std::map &entityControls, - [[maybe_unused]] const IPAOperationData &ipaConfig, - [[maybe_unused]] IPAOperationData *result) override {} - void mapBuffers([[maybe_unused]] const std::vector &buffers) override {} - void unmapBuffers([[maybe_unused]] const std::vector &ids) override {} - void processEvent([[maybe_unused]] const IPAOperationData &event) override {} - private: void initTrace(); - void trace(enum IPAOperationCode operation); + void trace(enum ipa::vimc::IPAOperationCode operation); int fd_; }; @@ -68,7 +56,7 @@ IPAVimc::~IPAVimc() int IPAVimc::init(const IPASettings &settings) { - trace(IPAOperationInit); + trace(ipa::vimc::IPAOperationInit); LOG(IPAVimc, Debug) << "initializing vimc IPA with configuration file " @@ -83,10 +71,9 @@ int IPAVimc::init(const IPASettings &settings) return 0; } -int IPAVimc::start([[maybe_unused]] const IPAOperationData &data, - [[maybe_unused]] IPAOperationData *result) +int IPAVimc::start() { - trace(IPAOperationStart); + trace(ipa::vimc::IPAOperationStart); LOG(IPAVimc, Debug) << "start vimc IPA!"; @@ -95,7 +82,7 @@ int IPAVimc::start([[maybe_unused]] const IPAOperationData &data, void IPAVimc::stop() { - trace(IPAOperationStop); + trace(ipa::vimc::IPAOperationStop); LOG(IPAVimc, Debug) << "stop vimc IPA!"; } @@ -103,11 +90,11 @@ void IPAVimc::stop() void IPAVimc::initTrace() { struct stat fifoStat; - int ret = stat(VIMC_IPA_FIFO_PATH, &fifoStat); + int ret = stat(ipa::vimc::VimcIPAFIFOPath.c_str(), &fifoStat); if (ret) return; - ret = ::open(VIMC_IPA_FIFO_PATH, O_WRONLY); + ret = ::open(ipa::vimc::VimcIPAFIFOPath.c_str(), O_WRONLY); if (ret < 0) { ret = errno; LOG(IPAVimc, Error) << "Failed to open vimc IPA test FIFO: " @@ -118,7 +105,7 @@ void IPAVimc::initTrace() fd_ = ret; } -void IPAVimc::trace(enum IPAOperationCode operation) +void IPAVimc::trace(enum ipa::vimc::IPAOperationCode operation) { if (fd_ < 0) return; @@ -143,9 +130,9 @@ const struct IPAModuleInfo ipaModuleInfo = { "vimc", }; -struct ipa_context *ipaCreate() +IPAInterface *ipaCreate() { - return new IPAInterfaceWrapper(std::make_unique()); + return new IPAVimc(); } } diff --git a/src/libcamera/ipa_context_wrapper.cpp b/src/libcamera/ipa_context_wrapper.cpp deleted file mode 100644 index 19c44ad8..00000000 --- a/src/libcamera/ipa_context_wrapper.cpp +++ /dev/null @@ -1,298 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_context_wrapper.cpp - Image Processing Algorithm context wrapper - */ - -#include "libcamera/internal/ipa_context_wrapper.h" - -#include - -#include - -#include "libcamera/internal/byte_stream_buffer.h" -#include "libcamera/internal/camera_sensor.h" -#include "libcamera/internal/utils.h" - -/** - * \file ipa_context_wrapper.h - * \brief Image Processing Algorithm context wrapper - */ - -namespace libcamera { - -/** - * \class IPAContextWrapper - * \brief Wrap an ipa_context and expose it as an IPAInterface - * - * The IPAContextWrapper class wraps an ipa_context, provided by an IPA module, and - * exposes an IPAInterface. This mechanism is used for IPAs that are not - * isolated in a separate process to allow direct calls from pipeline handler - * using the IPAInterface API instead of the lower-level ipa_context API. - * - * The IPAInterface methods are converted to the ipa_context API by translating - * all C++ arguments into plain C structures or byte arrays that contain no - * pointer, as required by the ipa_context API. - */ - -/** - * \brief Construct an IPAContextWrapper instance that wraps the \a context - * \param[in] context The IPA module context - * - * Ownership of the \a context is passed to the IPAContextWrapper. The context remains - * valid for the whole lifetime of the wrapper and is destroyed automatically - * with it. - */ -IPAContextWrapper::IPAContextWrapper(struct ipa_context *context) - : ctx_(context), intf_(nullptr) -{ - if (!ctx_) - return; - - bool forceCApi = !!utils::secure_getenv("LIBCAMERA_IPA_FORCE_C_API"); - - if (!forceCApi && ctx_ && ctx_->ops->get_interface) { - intf_ = reinterpret_cast(ctx_->ops->get_interface(ctx_)); - intf_->queueFrameAction.connect(this, &IPAContextWrapper::doQueueFrameAction); - return; - } - - ctx_->ops->register_callbacks(ctx_, &IPAContextWrapper::callbacks_, - this); -} - -IPAContextWrapper::~IPAContextWrapper() -{ - if (!ctx_) - return; - - ctx_->ops->destroy(ctx_); -} - -int IPAContextWrapper::init(const IPASettings &settings) -{ - if (intf_) - return intf_->init(settings); - - if (!ctx_) - return 0; - - struct ipa_settings c_settings; - c_settings.configuration_file = settings.configurationFile.c_str(); - - ctx_->ops->init(ctx_, &c_settings); - - return 0; -} - -int IPAContextWrapper::start(const IPAOperationData &data, - IPAOperationData *result) -{ - if (intf_) - return intf_->start(data, result); - - if (!ctx_) - return 0; - - return ctx_->ops->start(ctx_); -} - -void IPAContextWrapper::stop() -{ - if (intf_) - return intf_->stop(); - - if (!ctx_) - return; - - ctx_->ops->stop(ctx_); -} - -void IPAContextWrapper::configure(const CameraSensorInfo &sensorInfo, - const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *result) -{ - if (intf_) - return intf_->configure(sensorInfo, streamConfig, - entityControls, ipaConfig, result); - - if (!ctx_) - return; - - serializer_.reset(); - - /* Translate the camera sensor info. */ - struct ipa_sensor_info sensor_info = {}; - sensor_info.model = sensorInfo.model.c_str(); - sensor_info.bits_per_pixel = sensorInfo.bitsPerPixel; - sensor_info.active_area.width = sensorInfo.activeAreaSize.width; - sensor_info.active_area.height = sensorInfo.activeAreaSize.height; - sensor_info.analog_crop.left = sensorInfo.analogCrop.x; - sensor_info.analog_crop.top = sensorInfo.analogCrop.y; - sensor_info.analog_crop.width = sensorInfo.analogCrop.width; - sensor_info.analog_crop.height = sensorInfo.analogCrop.height; - sensor_info.output_size.width = sensorInfo.outputSize.width; - sensor_info.output_size.height = sensorInfo.outputSize.height; - sensor_info.pixel_rate = sensorInfo.pixelRate; - sensor_info.line_length = sensorInfo.lineLength; - - /* Translate the IPA stream configurations map. */ - struct ipa_stream c_streams[streamConfig.size()]; - - unsigned int i = 0; - for (const auto &stream : streamConfig) { - struct ipa_stream *c_stream = &c_streams[i]; - unsigned int id = stream.first; - const IPAStream &ipaStream = stream.second; - - c_stream->id = id; - c_stream->pixel_format = ipaStream.pixelFormat; - c_stream->width = ipaStream.size.width; - c_stream->height = ipaStream.size.height; - - ++i; - } - - /* Translate the IPA entity controls map. */ - struct ipa_control_info_map c_info_maps[entityControls.size()]; - std::vector> data(entityControls.size()); - - i = 0; - for (const auto &info : entityControls) { - struct ipa_control_info_map &c_info_map = c_info_maps[i]; - unsigned int id = info.first; - const ControlInfoMap &infoMap = info.second; - - size_t infoMapSize = serializer_.binarySize(infoMap); - data[i].resize(infoMapSize); - ByteStreamBuffer byteStream(data[i].data(), data[i].size()); - serializer_.serialize(infoMap, byteStream); - - c_info_map.id = id; - c_info_map.data = byteStream.base(); - c_info_map.size = byteStream.size(); - - ++i; - } - - /* \todo Translate the ipaConfig and reponse */ - ctx_->ops->configure(ctx_, &sensor_info, c_streams, streamConfig.size(), - c_info_maps, entityControls.size()); -} - -void IPAContextWrapper::mapBuffers(const std::vector &buffers) -{ - if (intf_) - return intf_->mapBuffers(buffers); - - if (!ctx_) - return; - - struct ipa_buffer c_buffers[buffers.size()]; - - for (unsigned int i = 0; i < buffers.size(); ++i) { - struct ipa_buffer &c_buffer = c_buffers[i]; - const IPABuffer &buffer = buffers[i]; - const std::vector &planes = buffer.planes; - - c_buffer.id = buffer.id; - c_buffer.num_planes = planes.size(); - - for (unsigned int j = 0; j < planes.size(); ++j) { - const FrameBuffer::Plane &plane = planes[j]; - c_buffer.planes[j].dmabuf = plane.fd.fd(); - c_buffer.planes[j].length = plane.length; - } - } - - ctx_->ops->map_buffers(ctx_, c_buffers, buffers.size()); -} - -void IPAContextWrapper::unmapBuffers(const std::vector &ids) -{ - if (intf_) - return intf_->unmapBuffers(ids); - - if (!ctx_) - return; - - ctx_->ops->unmap_buffers(ctx_, ids.data(), ids.size()); -} - -void IPAContextWrapper::processEvent(const IPAOperationData &data) -{ - if (intf_) - return intf_->processEvent(data); - - if (!ctx_) - return; - - struct ipa_operation_data c_data; - c_data.operation = data.operation; - c_data.data = data.data.data(); - c_data.num_data = data.data.size(); - - struct ipa_control_list control_lists[data.controls.size()]; - c_data.lists = control_lists; - c_data.num_lists = data.controls.size(); - - std::size_t listsSize = 0; - for (const auto &list : data.controls) - listsSize += serializer_.binarySize(list); - - std::vector binaryData(listsSize); - ByteStreamBuffer byteStreamBuffer(binaryData.data(), listsSize); - - unsigned int i = 0; - for (const auto &list : data.controls) { - struct ipa_control_list &c_list = control_lists[i]; - c_list.size = serializer_.binarySize(list); - ByteStreamBuffer b = byteStreamBuffer.carveOut(c_list.size); - - serializer_.serialize(list, b); - - c_list.data = b.base(); - } - - ctx_->ops->process_event(ctx_, &c_data); -} - -void IPAContextWrapper::doQueueFrameAction(unsigned int frame, - const IPAOperationData &data) -{ - IPAInterface::queueFrameAction.emit(frame, data); -} - -void IPAContextWrapper::queue_frame_action(void *ctx, unsigned int frame, - struct ipa_operation_data &data) -{ - IPAContextWrapper *_this = static_cast(ctx); - IPAOperationData opData; - - opData.operation = data.operation; - for (unsigned int i = 0; i < data.num_data; ++i) - opData.data.push_back(data.data[i]); - - for (unsigned int i = 0; i < data.num_lists; ++i) { - const struct ipa_control_list &c_list = data.lists[i]; - ByteStreamBuffer b(c_list.data, c_list.size); - opData.controls.push_back(_this->serializer_.deserialize(b)); - } - - _this->doQueueFrameAction(frame, opData); -} - -#ifndef __DOXYGEN__ -/* - * This construct confuses Doxygen and makes it believe that all members of the - * operations is a member of IPAContextWrapper. It must thus be hidden. - */ -const struct ipa_callback_ops IPAContextWrapper::callbacks_ = { - .queue_frame_action = &IPAContextWrapper::queue_frame_action, -}; -#endif - -} /* namespace libcamera */ diff --git a/src/libcamera/ipa_interface.cpp b/src/libcamera/ipa_interface.cpp index 5be6f787..c44581b2 100644 --- a/src/libcamera/ipa_interface.cpp +++ b/src/libcamera/ipa_interface.cpp @@ -15,371 +15,51 @@ * an Image Processing Algorithm (IPA) module. An IPA module is developed for a * specific pipeline handler and each pipeline handler may be compatible with * multiple IPA implementations, both open and closed source. To support this, - * libcamera communicates with IPA modules through a standard plain C interface. + * libcamera communicates with IPA modules through a per-pipeline C++ interface. * - * IPA modules shall expose a public function named ipaCreate() with the - * following prototype. + * IPA modules shall provide an ipaCreate() function exported as a public C + * symbol with the following prototype: * * \code{.c} - * struct ipa_context *ipaCreate(); + * IPAInterface *ipaCreate(); * \endcode * - * The ipaCreate() function creates an instance of an IPA context, which models + * The ipaCreate() function creates an instance of an IPA interface, which models * a context of execution for the IPA. IPA modules shall support creating one * context per camera, as required by their associated pipeline handler. * - * The IPA module context operations are defined in the struct ipa_context_ops. - * They model a low-level interface to configure the IPA, notify it of events, - * and receive IPA actions through callbacks. An IPA module stores a pointer to - * the operations corresponding to its context in the ipa_context::ops field. - * That pointer is immutable for the lifetime of the context, and may differ - * between different contexts created by the same IPA module. + * The IPA module interface operations are defined in the mojom file + * corresponding to the pipeline handler, in + * include/libcamera/ipa/{pipeline_name}.mojom. * - * The IPA interface defines base data types and functions to exchange data. On - * top of this, each pipeline handler is responsible for defining the set of - * events and actions used to communicate with their IPA. These are collectively - * referred to as IPA operations and define the pipeline handler-specific IPA - * protocol. Each operation defines the data that it carries, and how that data - * is encoded in the ipa_context_ops functions arguments. + * The IPA interface is specific to each pipeline handler. The pipeline handlers + * define a set of operations used to communicate with their IPA modules. The + * operations, along with the data structures they use, are collectively + * referred to as the IPA protocol. + * + * The IPA protocol is defined using the + * Mojo interface definition language, + * in a Mojo module file stored in include/libcamera/ipa/{pipeline_name}.mojom. + * The Mojo module contains two Mojo interfaces: IPAInterface defines the + * operations exposed by the IPA and called by the pipeline handler, and + * IPAEventInterface defines the events generated by the IPA and received by the + * pipeline handler. * * \todo Add reference to how pipelines shall document their protocol. * * IPAs can be isolated in a separate process. This implies that arguments to - * the IPA interface functions may need to be transferred over IPC. All - * arguments use Plain Old Data types and are documented either in the form of C - * data types, or as a textual description of byte arrays for types that can't - * be expressed using C data types (such as arrays of mixed data types). IPA - * modules can thus use the C API without calling into libcamera to access the - * data passed to the IPA context operations. + * the IPA interface functions may need to be transferred over IPC. An IPA + * proxy is auto-generated based on the mojom file, which abstracts away the + * (de)serialization from the pipeline handler and the IPA implementation. Thus + * any C++ structure that is defined in the mojom file, or the C++ libcamera + * objects that are listed in core.mojom, can be used directly. * * Due to IPC, synchronous communication between pipeline handlers and IPAs can - * be costly. For that reason, the interface operates asynchronously. This - * implies that methods don't return a status, and that all methods may copy - * their arguments. - * - * The IPAInterface class is a C++ representation of the ipa_context_ops, using - * C++ data classes provided by libcamera. This is the API exposed to pipeline - * handlers to communicate with IPA modules. IPA modules may use the - * IPAInterface API internally if they want to benefit from the data and helper - * classes offered by libcamera. - * - * When an IPA module is loaded directly into the libcamera process and uses - * the IPAInterface API internally, short-circuiting the path to the - * ipa_context_ops and back to IPAInterface is desirable. To support this, IPA - * modules may implement the ipa_context_ops::get_interface function to return a - * pointer to their internal IPAInterface. - */ - -/** - * \struct ipa_context - * \brief IPA module context of execution - * - * This structure models a context of execution for an IPA module. It is - * instantiated by the IPA module ipaCreate() function. IPA modules allocate - * context instances in an implementation-defined way, contexts shall thus be - * destroyed using the ipa_operation::destroy function only. - * - * The ipa_context structure provides a pointer to the IPA context operations. - * It shall otherwise be treated as a constant black-box cookie and passed - * unmodified to the functions defined in struct ipa_context_ops. - * - * IPA modules are expected to extend struct ipa_context by inheriting from it, - * either through structure embedding to model inheritance in plain C, or - * through C++ class inheritance. A simple example of the latter is available - * in the IPAContextWrapper class implementation. - * - * \var ipa_context::ops - * \brief The IPA context operations - */ - -/** - * \struct ipa_settings - * \brief IPA initialization settings for the IPA context operations - * \sa IPASettings - * - * \var ipa_settings::configuration_file - * \brief The name of the IPA configuration file (may be null or point to an - * empty string) - */ - -/** - * \struct ipa_sensor_info - * \brief Camera sensor information for the IPA context operations - * \sa libcamera::CameraSensorInfo - * - * \var ipa_sensor_info::model - * \brief The camera sensor model name - * \todo Remove this field as soon as no IPA depends on it anymore - * - * \var ipa_sensor_info::bits_per_pixel - * \brief The camera sensor image format bit depth - * \sa libcamera::CameraSensorInfo::bitsPerPixel - * - * \var ipa_sensor_info::active_area.width - * \brief The camera sensor pixel array active area width - * \sa libcamera::CameraSensorInfo::activeAreaSize - * - * \var ipa_sensor_info::active_area.height - * \brief The camera sensor pixel array active area height - * \sa libcamera::CameraSensorInfo::activeAreaSize - * - * \var ipa_sensor_info::active_area - * \brief The camera sensor pixel array active size - * \sa libcamera::CameraSensorInfo::activeAreaSize - * - * \var ipa_sensor_info::analog_crop.left - * \brief The left coordinate of the analog crop rectangle, relative to the - * pixel array active area - * \sa libcamera::CameraSensorInfo::analogCrop - * - * \var ipa_sensor_info::analog_crop.top - * \brief The top coordinate of the analog crop rectangle, relative to the pixel - * array active area - * \sa libcamera::CameraSensorInfo::analogCrop - * - * \var ipa_sensor_info::analog_crop.width - * \brief The horizontal size of the analog crop rectangle - * \sa libcamera::CameraSensorInfo::analogCrop - * - * \var ipa_sensor_info::analog_crop.height - * \brief The vertical size of the analog crop rectangle - * \sa libcamera::CameraSensorInfo::analogCrop - * - * \var ipa_sensor_info::analog_crop - * \brief The analog crop rectangle - * \sa libcamera::CameraSensorInfo::analogCrop - * - * \var ipa_sensor_info::output_size.width - * \brief The horizontal size of the output image - * \sa libcamera::CameraSensorInfo::outputSize - * - * \var ipa_sensor_info::output_size.height - * \brief The vertical size of the output image - * \sa libcamera::CameraSensorInfo::outputSize - * - * \var ipa_sensor_info::output_size - * \brief The size of the output image - * \sa libcamera::CameraSensorInfo::outputSize - * - * \var ipa_sensor_info::pixel_rate - * \brief The number of pixel produced in a second - * \sa libcamera::CameraSensorInfo::pixelRate - * - * \var ipa_sensor_info::line_length - * \brief The full line length, including blanking, in pixel units - * \sa libcamera::CameraSensorInfo::lineLength - */ - -/** - * \struct ipa_stream - * \brief Stream information for the IPA context operations - * - * \var ipa_stream::id - * \brief Identifier for the stream, defined by the IPA protocol - * - * \var ipa_stream::pixel_format - * \brief The stream pixel format, as defined by the PixelFormat class - * - * \var ipa_stream::width - * \brief The stream width in pixels - * - * \var ipa_stream::height - * \brief The stream height in pixels - */ - -/** - * \struct ipa_control_info_map - * \brief ControlInfoMap description for the IPA context operations - * - * \var ipa_control_info_map::id - * \brief Identifier for the ControlInfoMap, defined by the IPA protocol - * - * \var ipa_control_info_map::data - * \brief Pointer to a control packet for the ControlInfoMap - * \sa ipa_controls.h - * - * \var ipa_control_info_map::size - * \brief The size of the control packet in bytes - */ - -/** - * \struct ipa_buffer_plane - * \brief A plane for an ipa_buffer - * - * \var ipa_buffer_plane::dmabuf - * \brief The dmabuf file descriptor for the plane (-1 for unused planes) - * - * \var ipa_buffer_plane::length - * \brief The plane length in bytes (0 for unused planes) - */ - -/** - * \struct ipa_buffer - * \brief Buffer information for the IPA context operations - * - * \var ipa_buffer::id - * \brief The buffer unique ID (see \ref libcamera::IPABuffer::id) - * - * \var ipa_buffer::num_planes - * \brief The number of used planes in the ipa_buffer::planes array - * - * \var ipa_buffer::planes - * \brief The buffer planes (up to 3) - */ - -/** - * \struct ipa_control_list - * \brief ControlList description for the IPA context operations - * - * \var ipa_control_list::data - * \brief Pointer to a control packet for the ControlList - * \sa ipa_controls.h - * - * \var ipa_control_list::size - * \brief The size of the control packet in bytes - */ - -/** - * \struct ipa_operation_data - * \brief IPA operation data for the IPA context operations - * \sa libcamera::IPAOperationData - * - * \var ipa_operation_data::operation - * \brief IPA protocol operation - * - * \var ipa_operation_data::data - * \brief Pointer to the operation data array - * - * \var ipa_operation_data::num_data - * \brief Number of entries in the ipa_operation_data::data array - * - * \var ipa_operation_data::lists - * \brief Pointer to an array of ipa_control_list - * - * \var ipa_operation_data::num_lists - * \brief Number of entries in the ipa_control_list array - */ - -/** - * \struct ipa_callback_ops - * \brief IPA context operations as a set of function pointers - */ - -/** - * \var ipa_callback_ops::queue_frame_action - * \brief Queue an action associated with a frame to the pipeline handler - * \param[in] cb_ctx The callback context registered with - * ipa_context_ops::register_callbacks - * \param[in] frame The frame number - * - * \sa libcamera::IPAInterface::queueFrameAction - */ - -/** - * \struct ipa_context_ops - * \brief IPA context operations as a set of function pointers - * - * To allow for isolation of IPA modules in separate processes, the functions - * defined in the ipa_context_ops structure return only data related to the - * libcamera side of the operations. In particular, error related to the - * libcamera side of the IPC may be returned. Data returned by the IPA, - * including status information, shall be provided through callbacks from the - * IPA to libcamera. - */ - -/** - * \var ipa_context_ops::destroy - * \brief Destroy the IPA context created by the module's ipaCreate() function - * \param[in] ctx The IPA context - */ - -/** - * \var ipa_context_ops::get_interface - * \brief Retrieve the IPAInterface implemented by the ipa_context (optional) - * \param[in] ctx The IPA context - * - * IPA modules may implement this function to expose their internal - * IPAInterface, if any. When implemented, libcamera may at its sole discretion - * call it and then bypass the ipa_context_ops API by calling the IPAInterface - * methods directly. IPA modules shall still implement and support the full - * ipa_context_ops API. - */ - -/** - * \var ipa_context_ops::init - * \brief Initialise the IPA context - * \param[in] ctx The IPA context - * \param[in] settings The IPA initialization settings - * - * \sa libcamera::IPAInterface::init() - */ - -/** - * \var ipa_context_ops::start - * \brief Start the IPA context - * - * \sa libcamera::IPAInterface::start() - */ - -/** - * \var ipa_context_ops::stop - * \brief Stop the IPA context - * - * \sa libcamera::IPAInterface::stop() - */ - -/** - * \var ipa_context_ops::register_callbacks - * \brief Register callback operation from the IPA to the pipeline handler - * \param[in] ctx The IPA context - * \param[in] callback The IPA callback operations - * \param[in] cb_ctx The callback context, passed to all callback operations - */ - -/** - * \var ipa_context_ops::configure - * \brief Configure the IPA stream and sensor settings - * \param[in] ctx The IPA context - * \param[in] sensor_info Camera sensor information - * \param[in] streams Configuration of all active streams - * \param[in] num_streams The number of entries in the \a streams array - * \param[in] maps Controls provided by the pipeline entities - * \param[in] num_maps The number of entries in the \a maps array - * - * \sa libcamera::IPAInterface::configure() - */ - -/** - * \var ipa_context_ops::map_buffers - * \brief Map buffers shared between the pipeline handler and the IPA - * \param[in] ctx The IPA context - * \param[in] buffers The buffers to map - * \param[in] num_buffers The number of entries in the \a buffers array - * - * The dmabuf file descriptors provided in \a buffers are borrowed from the - * caller and are only guaranteed to be valid during the map_buffers() call. - * Should the callee need to store a copy of the file descriptors, it shall - * duplicate them first with ::%dup(). - * - * \sa libcamera::IPAInterface::mapBuffers() - */ - -/** - * \var ipa_context_ops::unmap_buffers - * \brief Unmap buffers shared by the pipeline to the IPA - * \param[in] ctx The IPA context - * \param[in] ids The IDs of the buffers to unmap - * \param[in] num_buffers The number of entries in the \a ids array - * - * \sa libcamera::IPAInterface::unmapBuffers() - */ - -/** - * \var ipa_context_ops::process_event - * \brief Process an event from the pipeline handler - * \param[in] ctx The IPA context - * - * \sa libcamera::IPAInterface::processEvent() + * be costly. For that reason, functions that cannot afford the high cost + * should be marked as [async] in the mojom file, and they will operate + * asynchronously. This implies that these methods don't return a status, and + * that all methods may copy their arguments. Synchronous functions are still + * allowed, but should be used with caution. */ /** @@ -387,128 +67,22 @@ * \brief Entry point to the IPA modules * * This function is the entry point to the IPA modules. It is implemented by - * every IPA module, and called by libcamera to create a new IPA context. + * every IPA module, and called by libcamera to create a new IPA interface + * instance. * - * \return A newly created IPA context + * \return A newly created IPA interface instance */ namespace libcamera { -/** - * \struct IPASettings - * \brief IPA interface initialization settings - * - * The IPASettings structure stores data passed to the IPAInterface::init() - * function. The data contains settings that don't depend on a particular camera - * or pipeline configuration and are valid for the whole life time of the IPA - * interface. - */ - -/** - * \var IPASettings::configurationFile - * \brief The name of the IPA configuration file - * - * This field may be an empty string if the IPA doesn't require a configuration - * file. - */ - -/** - * \struct IPAStream - * \brief Stream configuration for the IPA interface - * - * The IPAStream structure stores stream configuration parameters needed by the - * IPAInterface::configure() method. It mirrors the StreamConfiguration class - * that is not suitable for this purpose due to not being serializable. - */ - -/** - * \var IPAStream::pixelFormat - * \brief The stream pixel format - */ - -/** - * \var IPAStream::size - * \brief The stream size in pixels - */ - -/** - * \struct IPABuffer - * \brief Buffer information for the IPA interface - * - * The IPABuffer structure associates buffer memory with a unique ID. It is - * used to map buffers to the IPA with IPAInterface::mapBuffers(), after which - * buffers will be identified by their ID in the IPA interface. - */ - -/** - * \var IPABuffer::id - * \brief The buffer unique ID - * - * Buffers mapped to the IPA are identified by numerical unique IDs. The IDs - * are chosen by the pipeline handler to fulfil the following constraints: - * - * - IDs shall be positive integers different than zero - * - IDs shall be unique among all mapped buffers - * - * When buffers are unmapped with IPAInterface::unmapBuffers() their IDs are - * freed and may be reused for new buffer mappings. - */ - -/** - * \var IPABuffer::planes - * \brief The buffer planes description - * - * Stores the dmabuf handle and length for each plane of the buffer. - */ - -/** - * \struct IPAOperationData - * \brief Parameters for IPA operations - * - * The IPAOperationData structure carries parameters for the IPA operations - * performed through the IPAInterface::processEvent() method and the - * IPAInterface::queueFrameAction signal. - */ - -/** - * \var IPAOperationData::operation - * \brief IPA protocol operation - * - * The operation field describes which operation the receiver shall perform. It - * defines, through the IPA protocol, how the other fields of the structure are - * interpreted. The protocol freely assigns numerical values to operations. - */ - -/** - * \var IPAOperationData::data - * \brief Operation integer data - * - * The interpretation and position of different values in the array are defined - * by the IPA protocol. - */ - -/** - * \var IPAOperationData::controls - * \brief Operation controls data - * - * The interpretation and position of different values in the array are defined - * by the IPA protocol. - */ - /** * \class IPAInterface * \brief C++ Interface for IPA implementation * - * This pure virtual class defines a C++ API corresponding to the ipa_context, - * ipa_context_ops and ipa_callback_ops API. It is used by pipeline handlers to - * interact with IPA modules, and may be used internally in IPA modules if - * desired to benefit from the data and helper classes provided by libcamera. - * - * Functions defined in the ipa_context_ops structure are mapped to IPAInterface - * methods, while functions defined in the ipa_callback_ops are mapped to - * IPAInterface signals. As with the C API, the IPA C++ interface uses - * serializable data types only. It reuses structures defined by the C API, or - * defines corresponding classes using C++ containers when required. + * This pure virtual class defines a skeletal C++ API for IPA modules. + * Specializations of this class must be defined in a mojom file in + * include/libcamera/ipa/ (see the IPA Writers Guide for details + * on how to do so). * * Due to process isolation all arguments to the IPAInterface methods and * signals may need to be transferred over IPC. The class thus uses serializable @@ -516,147 +90,15 @@ namespace libcamera { * mirror core libcamera structures when the latter are not suitable, such as * IPAStream to carry StreamConfiguration data. * - * As for the functions defined in struct ipa_context_ops, the methods defined - * by this class shall not return data from the IPA. + * Custom data structures may also be defined in the mojom file, in which case + * the (de)serialization will automatically be generated. If any other libcamera + * structures are to be used as parameters, then a (de)serializer for them must + * be implemented in IPADataSerializer. * - * The pipeline handler shall use the IPAManager to locate a compatible + * The pipeline handlers shall use the IPAManager to locate a compatible * IPAInterface. The interface may then be used to interact with the IPA module. - */ - -/** - * \fn IPAInterface::init() - * \brief Initialise the IPAInterface - * \param[in] settings The IPA initialization settings - * - * This function initializes the IPA interface. It shall be called before any - * other function of the IPAInterface. The \a settings carry initialization - * parameters that are valid for the whole life time of the IPA interface. - */ - -/** - * \fn IPAInterface::start() - * \brief Start the IPA - * \param[in] data Protocol-specific data for the start operation - * \param[out] result Result of the start operation - * - * This method informs the IPA module that the camera is about to be started. - * The IPA module shall prepare any resources it needs to operate. - * - * The \a data and \a result parameters carry custom data passed by the - * pipeline handler to the IPA and back. The pipeline handler may set the \a - * result parameter to null if the IPA protocol doesn't need to pass a result - * back through the start() function. - * - * \return 0 on success or a negative error code otherwise - */ - -/** - * \fn IPAInterface::stop() - * \brief Stop the IPA - * - * This method informs the IPA module that the camera is stopped. The IPA module - * shall release resources prepared in start(). - */ - -/** - * \fn IPAInterface::configure() - * \brief Configure the IPA stream and sensor settings - * \param[in] sensorInfo Camera sensor information - * \param[in] streamConfig Configuration of all active streams - * \param[in] entityControls Controls provided by the pipeline entities - * \param[in] ipaConfig Pipeline-handler-specific configuration data - * \param[out] result Pipeline-handler-specific configuration result - * - * This method shall be called when the camera is started to inform the IPA of - * the camera's streams and the sensor settings. The meaning of the numerical - * keys in the \a streamConfig and \a entityControls maps is defined by the IPA - * protocol. - * - * The \a sensorInfo conveys information about the camera sensor settings that - * the pipeline handler has selected for the configuration. The IPA may use - * that information to tune its algorithms. - * - * The \a ipaConfig and \a result parameters carry custom data passed by the - * pipeline handler to the IPA and back. The pipeline handler may set the \a - * result parameter to null if the IPA protocol doesn't need to pass a result - * back through the configure() function. - */ - -/** - * \fn IPAInterface::mapBuffers() - * \brief Map buffers shared between the pipeline handler and the IPA - * \param[in] buffers List of buffers to map - * - * This method informs the IPA module of memory buffers set up by the pipeline - * handler that the IPA needs to access. It provides dmabuf file handles for - * each buffer, and associates the buffers with unique numerical IDs. - * - * IPAs shall map the dmabuf file handles to their address space and keep a - * cache of the mappings, indexed by the buffer numerical IDs. The IDs are used - * in all other IPA interface methods to refer to buffers, including the - * unmapBuffers() method. - * - * All buffers that the pipeline handler wishes to share with an IPA shall be - * mapped with this method. Buffers may be mapped all at once with a single - * call, or mapped and unmapped dynamically at runtime, depending on the IPA - * protocol. Regardless of the protocol, all buffers mapped at a given time - * shall have unique numerical IDs. - * - * The numerical IDs have no meaning defined by the IPA interface, and IPA - * protocols shall not give them any specific meaning either. They should be - * treated as opaque handles by IPAs, with the only exception that ID zero is - * invalid. - * - * \sa unmapBuffers() - * - * \todo Provide a generic implementation of mapBuffers and unmapBuffers for - * IPAs - */ - -/** - * \fn IPAInterface::unmapBuffers() - * \brief Unmap buffers shared by the pipeline to the IPA - * \param[in] ids List of buffer IDs to unmap - * - * This method removes mappings set up with mapBuffers(). Buffers may be - * unmapped all at once with a single call, or selectively at runtime, depending - * on the IPA protocol. Numerical IDs of unmapped buffers may be reused when - * mapping new buffers. - * - * \sa mapBuffers() - */ - -/** - * \fn IPAInterface::processEvent() - * \brief Process an event from the pipeline handler - * \param[in] data IPA operation data - * - * This operation is used by pipeline handlers to inform the IPA module of - * events that occurred during the on-going capture operation. - * - * The event notified by the pipeline handler with this method is handled by the - * IPA, which interprets the operation parameters according to the separately - * documented IPA protocol. - */ - -/** - * \var IPAInterface::queueFrameAction - * \brief Queue an action associated with a frame to the pipeline handler - * \param[in] frame The frame number for the action - * \param[in] data IPA operation data - * - * This signal is emitted when the IPA wishes to queue a FrameAction on the - * pipeline. The pipeline is still responsible for the scheduling of the action - * on its timeline. - * - * This signal is emitted by the IPA to queue an action to be executed by the - * pipeline handler on a frame. The type of action is identified by the - * \a data.operation field, as defined by the IPA protocol, and the rest of the - * \a data is interpreted accordingly. The pipeline handler shall queue the - * action and execute it as appropriate. * - * The signal is only emitted when the IPA is running, that is after start() and - * before stop() have been called. + * \todo Figure out how to generate IPAInterface documentation. */ } /* namespace libcamera */ diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index ad05b9c9..93d02d94 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -245,6 +245,7 @@ unsigned int IPAManager::addDir(const char *libDir, unsigned int maxDepth) } /** + * \fn IPAManager::createIPA() * \brief Create an IPA proxy that matches a given pipeline handler * \param[in] pipe The pipeline handler that wants a matching IPA proxy * \param[in] minVersion Minimum acceptable version of IPA module @@ -253,52 +254,6 @@ unsigned int IPAManager::addDir(const char *libDir, unsigned int maxDepth) * \return A newly created IPA proxy, or nullptr if no matching IPA module is * found or if the IPA proxy fails to initialize */ -std::unique_ptr IPAManager::createIPA(PipelineHandler *pipe, - uint32_t maxVersion, - uint32_t minVersion) -{ - IPAModule *m = nullptr; - - for (IPAModule *module : self_->modules_) { - if (module->match(pipe, minVersion, maxVersion)) { - m = module; - break; - } - } - - if (!m) - return nullptr; - - /* - * Load and run the IPA module in a thread if it has a valid signature, - * or isolate it in a separate process otherwise. - * - * \todo Implement a better proxy selection - */ - const char *proxyName = self_->isSignatureValid(m) - ? "IPAProxyThread" : "IPAProxyLinux"; - IPAProxyFactory *pf = nullptr; - - for (IPAProxyFactory *factory : IPAProxyFactory::factories()) { - if (!strcmp(factory->name().c_str(), proxyName)) { - pf = factory; - break; - } - } - - if (!pf) { - LOG(IPAManager, Error) << "Failed to get proxy factory"; - return nullptr; - } - - std::unique_ptr proxy = pf->create(m); - if (!proxy->isValid()) { - LOG(IPAManager, Error) << "Failed to load proxy"; - return nullptr; - } - - return proxy; -} bool IPAManager::isSignatureValid([[maybe_unused]] IPAModule *ipa) const { diff --git a/src/libcamera/ipa_module.cpp b/src/libcamera/ipa_module.cpp index de512a7f..f53e529b 100644 --- a/src/libcamera/ipa_module.cpp +++ b/src/libcamera/ipa_module.cpp @@ -391,13 +391,13 @@ const std::string &IPAModule::path() const /** * \brief Load the IPA implementation factory from the shared object * - * The IPA module shared object implements an ipa_context object to be used + * The IPA module shared object implements an IPAInterface object to be used * by pipeline handlers. This method loads the factory function from the - * shared object. Later, createContext() can be called to instantiate the - * ipa_context. + * shared object. Later, createInterface() can be called to instantiate the + * IPAInterface. * * This method only needs to be called successfully once, after which - * createContext() can be called as many times as ipa_context instances are + * createInterface() can be called as many times as IPAInterface instances are * needed. * * Calling this function on an invalid module (as returned by isValid()) is @@ -439,20 +439,18 @@ bool IPAModule::load() } /** - * \brief Instantiate an IPA context + * \brief Instantiate an IPA interface * * After loading the IPA module with load(), this method creates an instance of - * the IPA module context. Ownership of the context is passed to the caller, and - * the context shall be destroyed by calling the \ref ipa_context_ops::destroy - * "ipa_context::ops::destroy()" function. + * the IPA module interface. * * Calling this function on a module that has not yet been loaded, or an * invalid module (as returned by load() and isValid(), respectively) is * an error. * - * \return The IPA context on success, or nullptr on error + * \return The IPA interface on success, or nullptr on error */ -struct ipa_context *IPAModule::createContext() +IPAInterface *IPAModule::createInterface() { if (!valid_ || !loaded_) return nullptr; diff --git a/src/libcamera/ipa_proxy.cpp b/src/libcamera/ipa_proxy.cpp index ff4d7fd1..29c0e9e0 100644 --- a/src/libcamera/ipa_proxy.cpp +++ b/src/libcamera/ipa_proxy.cpp @@ -30,17 +30,11 @@ LOG_DEFINE_CATEGORY(IPAProxy) * \brief IPA Proxy * * Isolate IPA into separate process. - * - * Every subclass of proxy shall be registered with libcamera using - * the REGISTER_IPA_PROXY() macro. */ /** * \brief Construct an IPAProxy instance * \param[in] ipam The IPA module - * - * IPAProxy instances shall be constructed through the IPAProxyFactory::create() - * method implemented by the respective factories. */ IPAProxy::IPAProxy(IPAModule *ipam) : valid_(false), ipam_(ipam) @@ -145,16 +139,6 @@ std::string IPAProxy::configurationFile(const std::string &name) const return std::string(); } -/** - * \fn IPAProxy::stop() - * \brief Stop the IPA proxy - * - * This function stops the IPA and releases all the resources acquired by the - * proxy in start(). Calling stop() when the IPA proxy hasn't been started or - * has already been stopped is valid, the proxy shall treat this as a no-op and - * shall not forward the call to the IPA. - */ - /** * \brief Find a valid full path for a proxy worker for a given executable name * \param[in] file File name of proxy worker executable @@ -229,89 +213,4 @@ std::string IPAProxy::resolvePath(const std::string &file) const * construction. */ -/** - * \class IPAProxyFactory - * \brief Registration of IPAProxy classes and creation of instances - * - * To facilitate discovery and instantiation of IPAProxy classes, the - * IPAProxyFactory class maintains a registry of IPAProxy classes. Each - * IPAProxy subclass shall register itself using the REGISTER_IPA_PROXY() - * macro, which will create a corresponding instance of a IPAProxyFactory - * subclass and register it with the static list of factories. - */ - -/** - * \brief Construct a IPAProxy factory - * \param[in] name Name of the IPAProxy class - * - * Creating an instance of the factory registers is with the global list of - * factories, accessible through the factories() function. - * - * The factory \a name is used for debugging and IPAProxy matching purposes - * and shall be unique. - */ -IPAProxyFactory::IPAProxyFactory(const char *name) - : name_(name) -{ - registerType(this); -} - -/** - * \fn IPAProxyFactory::create() - * \brief Create an instance of the IPAProxy corresponding to the factory - * \param[in] ipam The IPA module - * - * This virtual function is implemented by the REGISTER_IPA_PROXY() macro. - * It creates a IPAProxy instance that isolates an IPA interface designated - * by the IPA module \a ipam. - * - * \return A pointer to a newly constructed instance of the IPAProxy subclass - * corresponding to the factory - */ - -/** - * \fn IPAProxyFactory::name() - * \brief Retrieve the factory name - * \return The factory name - */ - -/** - * \brief Add a IPAProxy class to the registry - * \param[in] factory Factory to use to construct the IPAProxy - * - * The caller is responsible to guarantee the uniqueness of the IPAProxy name. - */ -void IPAProxyFactory::registerType(IPAProxyFactory *factory) -{ - std::vector &factories = IPAProxyFactory::factories(); - - factories.push_back(factory); - - LOG(IPAProxy, Debug) - << "Registered proxy \"" << factory->name() << "\""; -} - -/** - * \brief Retrieve the list of all IPAProxy factories - * - * The static factories map is defined inside the function to ensure it gets - * initialized on first use, without any dependency on link order. - * - * \return The list of pipeline handler factories - */ -std::vector &IPAProxyFactory::factories() -{ - static std::vector factories; - return factories; -} - -/** - * \def REGISTER_IPA_PROXY - * \brief Register a IPAProxy with the IPAProxy factory - * \param[in] proxy Class name of IPAProxy derived class to register - * - * Register a proxy subclass with the factory and make it available to - * isolate IPA modules. - */ - } /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 6b5d126b..4b5e33ce 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -24,7 +24,6 @@ libcamera_sources = files([ 'formats.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', - 'ipa_context_wrapper.cpp', 'ipa_controls.cpp', 'ipa_data_serializer.cpp', 'ipa_interface.cpp', diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 61f7bf43..3e6b88af 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -14,7 +14,8 @@ #include #include #include -#include +#include +#include #include #include @@ -77,8 +78,11 @@ public: std::unique_ptr delayedCtrls_; IPU3Frames frameInfos_; + std::unique_ptr ipa_; + private: - void queueFrameAction(unsigned int id, const IPAOperationData &op); + void queueFrameAction(unsigned int id, + const ipa::ipu3::IPU3Action &action); }; class IPU3CameraConfiguration : public CameraConfiguration @@ -609,18 +613,12 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) for (const std::unique_ptr &buffer : imgu->paramBuffers_) { buffer->setCookie(ipaBufferId++); - ipaBuffers_.push_back({ - .id = buffer->cookie(), - .planes = buffer->planes() - }); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } for (const std::unique_ptr &buffer : imgu->statBuffers_) { buffer->setCookie(ipaBufferId++); - ipaBuffers_.push_back({ - .id = buffer->cookie(), - .planes = buffer->planes() - }); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } data->ipa_->mapBuffers(ipaBuffers_); @@ -650,16 +648,10 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *controls) { + std::map entityControls; IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; - - CameraSensorInfo sensorInfo = {}; - std::map streamConfig; - std::map entityControls; - IPAOperationData ipaConfig; - IPAOperationData result = {}; - int ret; /* Allocate buffers for internal pipeline usage. */ @@ -667,8 +659,7 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *con if (ret) return ret; - IPAOperationData ipaData = {}; - ret = data->ipa_->start(ipaData, nullptr); + ret = data->ipa_->start(); if (ret) goto error; @@ -684,24 +675,8 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *con if (ret) goto error; - /* Inform IPA of stream configuration and sensor controls. */ - ret = data->cio2_.sensor()->sensorInfo(&sensorInfo); - if (ret) - goto error; - - streamConfig[0] = { - .pixelFormat = data->outStream_.configuration().pixelFormat, - .size = data->outStream_.configuration().size, - }; - streamConfig[1] = { - .pixelFormat = data->vfStream_.configuration().pixelFormat, - .size = data->vfStream_.configuration().size, - }; - entityControls.emplace(0, data->cio2_.sensor()->controls()); - - data->ipa_->configure(sensorInfo, streamConfig, entityControls, - ipaConfig, &result); + data->ipa_->configure(entityControls); return 0; @@ -751,11 +726,11 @@ int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request) info->rawBuffer = rawBuffer; - IPAOperationData op; - op.operation = IPU3_IPA_EVENT_PROCESS_CONTROLS; - op.data = { info->id }; - op.controls = { request->controls() }; - data->ipa_->processEvent(op); + ipa::ipu3::IPU3Event ev; + ev.op = ipa::ipu3::EventProcessControls; + ev.frame = info->id; + ev.controls = request->controls(); + data->ipa_->processEvent(ev); return 0; } @@ -1048,7 +1023,7 @@ int PipelineHandlerIPU3::registerCameras() int IPU3CameraData::loadIPA() { - ipa_ = IPAManager::createIPA(pipe_, 1, 1); + ipa_ = IPAManager::createIPA(pipe_, 1, 1); if (!ipa_) return -ENOENT; @@ -1060,15 +1035,15 @@ int IPU3CameraData::loadIPA() } void IPU3CameraData::queueFrameAction(unsigned int id, - const IPAOperationData &action) + const ipa::ipu3::IPU3Action &action) { - switch (action.operation) { - case IPU3_IPA_ACTION_SET_SENSOR_CONTROLS: { - const ControlList &controls = action.controls[0]; + switch (action.op) { + case ipa::ipu3::ActionSetSensorControls: { + const ControlList &controls = action.controls; delayedCtrls_->push(controls); break; } - case IPU3_IPA_ACTION_PARAM_FILLED: { + case ipa::ipu3::ActionParamFilled: { IPU3Frames::Info *info = frameInfos_.find(id); if (!info) break; @@ -1090,13 +1065,13 @@ void IPU3CameraData::queueFrameAction(unsigned int id, break; } - case IPU3_IPA_ACTION_METADATA_READY: { + case ipa::ipu3::ActionMetadataReady: { IPU3Frames::Info *info = frameInfos_.find(id); if (!info) break; Request *request = info->request; - request->metadata() = action.controls[0]; + request->metadata() = action.controls; info->metadataProcessed = true; if (frameInfos_.tryComplete(info)) pipe_->completeRequest(request); @@ -1104,7 +1079,7 @@ void IPU3CameraData::queueFrameAction(unsigned int id, break; } default: - LOG(IPU3, Error) << "Unknown action " << action.operation; + LOG(IPU3, Error) << "Unknown action " << action.op; break; } } @@ -1172,10 +1147,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) if (request->findBuffer(&rawStream_)) pipe_->completeBuffer(request, buffer); - IPAOperationData op; - op.operation = IPU3_IPA_EVENT_FILL_PARAMS; - op.data = { info->id, info->paramBuffer->cookie() }; - ipa_->processEvent(op); + ipa::ipu3::IPU3Event ev; + ev.op = ipa::ipu3::EventFillParams; + ev.frame = info->id; + ev.bufferId = info->paramBuffer->cookie(); + ipa_->processEvent(ev); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) @@ -1202,10 +1178,11 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer) return; } - IPAOperationData op; - op.operation = IPU3_IPA_EVENT_STAT_READY; - op.data = { info->id, info->statBuffer->cookie() }; - ipa_->processEvent(op); + ipa::ipu3::IPU3Event ev; + ev.op = ipa::ipu3::EventStatReady; + ev.frame = info->id; + ev.bufferId = info->statBuffer->cookie(); + ipa_->processEvent(ev); } REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index e4764681..15aa600e 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -18,10 +18,13 @@ #include #include #include +#include +#include #include #include #include +#include #include #include "libcamera/internal/bayer_format.h" @@ -146,7 +149,11 @@ public: int loadIPA(); int configureIPA(const CameraConfiguration *config); - void queueFrameAction(unsigned int frame, const IPAOperationData &action); + void statsMetadataComplete(uint32_t bufferId, const ControlList &controls); + void runIsp(uint32_t bufferId); + void embeddedComplete(uint32_t bufferId); + void setIsp(const ControlList &controls); + void setDelayedControls(const ControlList &controls); /* bufferComplete signal handlers. */ void unicamBufferDequeue(FrameBuffer *buffer); @@ -159,6 +166,8 @@ public: void handleState(); void applyScalerCrop(const ControlList &controls); + std::unique_ptr ipa_; + std::unique_ptr sensor_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ RPi::Device unicam_; @@ -751,7 +760,7 @@ int PipelineHandlerRPi::exportFrameBuffers([[maybe_unused]] Camera *camera, Stre return ret; } -int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *controls) +int PipelineHandlerRPi::start(Camera *camera, ControlList *controls) { RPiCameraData *data = cameraData(camera); int ret; @@ -769,30 +778,20 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont data->applyScalerCrop(*controls); /* Start the IPA. */ - IPAOperationData ipaData = {}; - IPAOperationData result = {}; - if (controls) { - ipaData.operation = RPi::IPA_CONFIG_STARTUP_CTRLS; - ipaData.controls.emplace_back(*controls); - } - ret = data->ipa_->start(ipaData, &result); - if (ret) { - LOG(RPI, Error) - << "Failed to start IPA for " << camera->id(); - stop(camera); - return ret; - } + ipa::rpi::StartControls ipaData; + ipa::rpi::StartControls result; + if (controls) + ipaData.controls = *controls; + data->ipa_->start(ipaData, &result); /* Apply any gain/exposure settings that the IPA may have passed back. */ - if (result.operation & RPi::IPA_RESULT_SENSOR_CTRLS) { - ControlList &ctrls = result.controls[0]; + if (!result.controls.empty()) { + ControlList &ctrls = result.controls; data->unicam_[Unicam::Image].dev()->setControls(&ctrls); } - if (result.operation & RPi::IPA_RESULT_DROP_FRAMES) { - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = result.data[0]; - } + /* Configure the number of dropped frames required on startup. */ + data->dropFrameCount_ = result.dropFrameCount; /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); @@ -1115,8 +1114,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * Pass the stats and embedded data buffers to the IPA. No other * buffers need to be passed. */ - mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPi::BufferMask::STATS); - mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPi::BufferMask::EMBEDDED_DATA); + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), ipa::rpi::MaskStats); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), ipa::rpi::MaskEmbeddedData); return 0; } @@ -1133,8 +1132,8 @@ void PipelineHandlerRPi::mapBuffers(Camera *camera, const RPi::BufferMap &buffer * handler and the IPA. */ for (auto const &it : buffers) { - ipaBuffers.push_back({ .id = mask | it.first, - .planes = it.second->planes() }); + ipaBuffers.push_back(IPABuffer(mask | it.first, + it.second->planes())); data->ipaBuffers_.insert(mask | it.first); } @@ -1165,15 +1164,18 @@ void RPiCameraData::frameStarted(uint32_t sequence) int RPiCameraData::loadIPA() { - ipa_ = IPAManager::createIPA(pipe_, 1, 1); + ipa_ = IPAManager::createIPA(pipe_, 1, 1); + if (!ipa_) return -ENOENT; - ipa_->queueFrameAction.connect(this, &RPiCameraData::queueFrameAction); + ipa_->statsMetadataComplete.connect(this, &RPiCameraData::statsMetadataComplete); + ipa_->runIsp.connect(this, &RPiCameraData::runIsp); + ipa_->embeddedComplete.connect(this, &RPiCameraData::embeddedComplete); + ipa_->setIsp.connect(this, &RPiCameraData::setIsp); + ipa_->setDelayedControls.connect(this, &RPiCameraData::setDelayedControls); - IPASettings settings{ - .configurationFile = ipa_->configurationFile(sensor_->model() + ".json") - }; + IPASettings settings(ipa_->configurationFile(sensor_->model() + ".json")); return ipa_->init(settings); } @@ -1185,8 +1187,8 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) static_cast(config); std::map streamConfig; - std::map entityControls; - IPAOperationData ipaConfig = {}; + std::map entityControls; + ipa::rpi::ConfigInput ipaConfig; /* Get the device format to pass to the IPA. */ V4L2DeviceFormat sensorFormat; @@ -1195,10 +1197,9 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) unsigned int i = 0; for (auto const &stream : isp_) { if (stream.isExternal()) { - streamConfig[i++] = { - .pixelFormat = stream.configuration().pixelFormat, - .size = stream.configuration().size - }; + streamConfig[i++] = IPAStream( + stream.configuration().pixelFormat, + stream.configuration().size); } } @@ -1206,17 +1207,20 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) entityControls.emplace(1, isp_[Isp::Input].dev()->controls()); /* Always send the user transform to the IPA. */ - ipaConfig.data = { static_cast(config->transform) }; + ipaConfig.transform = static_cast(config->transform); /* Allocate the lens shading table via dmaHeap and pass to the IPA. */ if (!lsTable_.isValid()) { - lsTable_ = dmaHeap_.alloc("ls_grid", RPi::MaxLsGridSize); + lsTable_ = dmaHeap_.alloc("ls_grid", ipa::rpi::MaxLsGridSize); if (!lsTable_.isValid()) return -ENOMEM; /* Allow the IPA to mmap the LS table via the file descriptor. */ - ipaConfig.operation = RPi::IPA_CONFIG_LS_TABLE; - ipaConfig.data.push_back(static_cast(lsTable_.fd())); + /* + * \todo Investigate if mapping the lens shading table buffer + * could be handled with mapBuffers(). + */ + ipaConfig.lsTableHandle = lsTable_; } /* We store the CameraSensorInfo for digital zoom calculations. */ @@ -1227,35 +1231,34 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) } /* Ready the IPA - it must know about the sensor resolution. */ - IPAOperationData result = {}; + ipa::rpi::ConfigOutput result; ipa_->configure(sensorInfo_, streamConfig, entityControls, ipaConfig, - &result); + &result, &ret); - if (result.operation & RPi::IPA_RESULT_CONFIG_FAILED) { + if (ret < 0) { LOG(RPI, Error) << "IPA configuration failed!"; return -EPIPE; } - unsigned int resultIdx = 0; - if (result.operation & RPi::IPA_RESULT_SENSOR_PARAMS) { + if (result.params & ipa::rpi::ConfigStaggeredWrite) { /* * Setup our delayed control writer with the sensor default * gain and exposure delays. */ std::unordered_map delays = { - { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, - { V4L2_CID_EXPOSURE, result.data[resultIdx++] }, - { V4L2_CID_VBLANK, result.data[resultIdx++] } + { V4L2_CID_ANALOGUE_GAIN, result.sensorConfig.gainDelay }, + { V4L2_CID_EXPOSURE, result.sensorConfig.exposureDelay }, + { V4L2_CID_VBLANK, result.sensorConfig.vblank } }; delayedCtrls_ = std::make_unique(unicam_[Unicam::Image].dev(), delays); - sensorMetadata_ = result.data[resultIdx++]; + sensorMetadata_ = result.sensorConfig.sensorMetadata; } - if (result.operation & RPi::IPA_RESULT_SENSOR_CTRLS) { - ControlList &ctrls = result.controls[0]; + if (!result.controls.empty()) { + ControlList &ctrls = result.controls; unicam_[Unicam::Image].dev()->setControls(&ctrls); } @@ -1275,90 +1278,86 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) return 0; } -void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, - const IPAOperationData &action) +void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList &controls) { + if (state_ == State::Stopped) + handleState(); + + FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); + + handleStreamBuffer(buffer, &isp_[Isp::Stats]); + + /* Fill the Request metadata buffer with what the IPA has provided */ + Request *request = requestQueue_.front(); + request->metadata() = std::move(controls); + /* - * The following actions can be handled when the pipeline handler is in - * a stopped state. + * Also update the ScalerCrop in the metadata with what we actually + * used. But we must first rescale that from ISP (camera mode) pixels + * back into sensor native pixels. + * + * Sending this information on every frame may be helpful. */ - switch (action.operation) { - case RPi::IPA_ACTION_SET_DELAYED_CTRLS: { - const ControlList &controls = action.controls[0]; - if (!delayedCtrls_->push(controls)) - LOG(RPI, Error) << "Failed to set delayed controls"; - goto done; + if (updateScalerCrop_) { + updateScalerCrop_ = false; + scalerCrop_ = ispCrop_.scaledBy(sensorInfo_.analogCrop.size(), + sensorInfo_.outputSize); + scalerCrop_.translateBy(sensorInfo_.analogCrop.topLeft()); } + request->metadata().set(controls::ScalerCrop, scalerCrop_); - case RPi::IPA_ACTION_V4L2_SET_ISP: { - ControlList controls = action.controls[0]; - isp_[Isp::Input].dev()->setControls(&controls); - goto done; - } - } + state_ = State::IpaComplete; - if (state_ == State::Stopped) - goto done; + handleState(); +} - /* - * The following actions must not be handled when the pipeline handler - * is in a stopped state. - */ - switch (action.operation) { - case RPi::IPA_ACTION_STATS_METADATA_COMPLETE: { - unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); +void RPiCameraData::runIsp(uint32_t bufferId) +{ + if (state_ == State::Stopped) + handleState(); - handleStreamBuffer(buffer, &isp_[Isp::Stats]); + FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); - /* Fill the Request metadata buffer with what the IPA has provided */ - Request *request = requestQueue_.front(); - request->metadata() = std::move(action.controls[0]); + LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId + << ", timestamp: " << buffer->metadata().timestamp; - /* - * Also update the ScalerCrop in the metadata with what we actually - * used. But we must first rescale that from ISP (camera mode) pixels - * back into sensor native pixels. - * - * Sending this information on every frame may be helpful. - */ - if (updateScalerCrop_) { - updateScalerCrop_ = false; - scalerCrop_ = ispCrop_.scaledBy(sensorInfo_.analogCrop.size(), - sensorInfo_.outputSize); - scalerCrop_.translateBy(sensorInfo_.analogCrop.topLeft()); - } - request->metadata().set(controls::ScalerCrop, scalerCrop_); + isp_[Isp::Input].queueBuffer(buffer); + ispOutputCount_ = 0; + handleState(); +} - state_ = State::IpaComplete; - break; - } +void RPiCameraData::embeddedComplete(uint32_t bufferId) +{ + if (state_ == State::Stopped) + handleState(); - case RPi::IPA_ACTION_EMBEDDED_COMPLETE: { - unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); - handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); - break; - } + FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); + handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); + handleState(); +} - case RPi::IPA_ACTION_RUN_ISP: { - unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); +void RPiCameraData::setIsp(const ControlList &controls) +{ + ControlList ctrls = controls; - LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId - << ", timestamp: " << buffer->metadata().timestamp; + Span s = + ctrls.get(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING).data(); + bcm2835_isp_lens_shading ls = + *reinterpret_cast(s.data()); + ls.dmabuf = lsTable_.fd(); - isp_[Isp::Input].queueBuffer(buffer); - ispOutputCount_ = 0; - break; - } + ControlValue c(Span{ reinterpret_cast(&ls), + sizeof(ls) }); + ctrls.set(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING, c); - default: - LOG(RPI, Error) << "Unknown action " << action.operation; - break; - } + isp_[Isp::Input].dev()->setControls(&ctrls); + handleState(); +} -done: +void RPiCameraData::setDelayedControls(const ControlList &controls) +{ + if (!delayedCtrls_->push(controls)) + LOG(RPI, Error) << "V4L2 staggered set failed"; handleState(); } @@ -1456,10 +1455,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) * application until after the IPA signals so. */ if (stream == &isp_[Isp::Stats]) { - IPAOperationData op; - op.operation = RPi::IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPi::BufferMask::STATS | static_cast(index) }; - ipa_->processEvent(op); + ipa_->signalStatReady(ipa::rpi::MaskStats | static_cast(index)); } else { /* Any other ISP output can be handed back to the application now. */ handleStreamBuffer(buffer, stream); @@ -1563,7 +1559,7 @@ void RPiCameraData::handleExternalBuffer(FrameBuffer *buffer, RPi::Stream *strea { unsigned int id = stream->getBufferId(buffer); - if (!(id & RPi::BufferMask::EXTERNAL_BUFFER)) + if (!(id & ipa::rpi::MaskExternalBuffer)) return; /* Stop the Stream object from tracking the buffer. */ @@ -1663,7 +1659,6 @@ void RPiCameraData::applyScalerCrop(const ControlList &controls) void RPiCameraData::tryRunPipeline() { FrameBuffer *bayerBuffer, *embeddedBuffer; - IPAOperationData op; /* If any of our request or buffer queues are empty, we cannot proceed. */ if (state_ != State::Idle || requestQueue_.empty() || @@ -1684,9 +1679,7 @@ void RPiCameraData::tryRunPipeline() * queue the ISP output buffer listed in the request to start the HW * pipeline. */ - op.operation = RPi::IPA_EVENT_QUEUE_REQUEST; - op.controls = { request->controls() }; - ipa_->processEvent(op); + ipa_->signalQueueRequest(request->controls()); /* Set our state to say the pipeline is active. */ state_ = State::Busy; @@ -1694,14 +1687,14 @@ void RPiCameraData::tryRunPipeline() unsigned int bayerId = unicam_[Unicam::Image].getBufferId(bayerBuffer); unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer); - LOG(RPI, Debug) << "Signalling RPi::IPA_EVENT_SIGNAL_ISP_PREPARE:" + LOG(RPI, Debug) << "Signalling signalIspPrepare:" << " Bayer buffer id: " << bayerId << " Embedded buffer id: " << embeddedId; - op.operation = RPi::IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPi::BufferMask::EMBEDDED_DATA | embeddedId, - RPi::BufferMask::BAYER_DATA | bayerId }; - ipa_->processEvent(op); + ipa::rpi::ISPConfig ispPrepare; + ispPrepare.embeddedbufferId = ipa::rpi::MaskEmbeddedData | embeddedId; + ispPrepare.bayerbufferId = ipa::rpi::MaskBayerData | bayerId; + ipa_->signalIspPrepare(ispPrepare); } bool RPiCameraData::findMatchingBuffers(FrameBuffer *&bayerBuffer, FrameBuffer *&embeddedBuffer) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 3a5dadab..496dd36f 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -6,6 +6,8 @@ */ #include "rpi_stream.h" +#include + #include "libcamera/internal/log.h" namespace libcamera { @@ -70,7 +72,7 @@ int Stream::getBufferId(FrameBuffer *buffer) const void Stream::setExternalBuffer(FrameBuffer *buffer) { - bufferMap_.emplace(RPi::BufferMask::EXTERNAL_BUFFER | id_.get(), buffer); + bufferMap_.emplace(ipa::rpi::MaskExternalBuffer | id_.get(), buffer); } void Stream::removeExternalBuffer(FrameBuffer *buffer) @@ -78,7 +80,7 @@ void Stream::removeExternalBuffer(FrameBuffer *buffer) int id = getBufferId(buffer); /* Ensure we have this buffer in the stream, and it is marked external. */ - ASSERT(id != -1 && (id & RPi::BufferMask::EXTERNAL_BUFFER)); + ASSERT(id != -1 && (id & ipa::rpi::MaskExternalBuffer)); bufferMap_.erase(id); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 0b502f64..701110d0 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -13,6 +13,7 @@ #include #include +#include #include #include "libcamera/internal/v4l2_videodevice.h" @@ -31,13 +32,13 @@ class Stream : public libcamera::Stream { public: Stream() - : id_(RPi::BufferMask::ID) + : id_(ipa::rpi::MaskID) { } Stream(const char *name, MediaEntity *dev, bool importOnly = false) : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)), id_(RPi::BufferMask::ID) + dev_(std::make_unique(dev)), id_(ipa::rpi::MaskID) { } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 7cb89eb0..a794501a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -18,7 +18,9 @@ #include #include #include -#include +#include +#include +#include #include #include @@ -96,9 +98,11 @@ public: RkISP1MainPath *mainPath_; RkISP1SelfPath *selfPath_; + std::unique_ptr ipa_; + private: void queueFrameAction(unsigned int frame, - const IPAOperationData &action); + const ipa::rkisp1::RkISP1Action &action); void metadataReady(unsigned int frame, const ControlList &metadata); }; @@ -298,7 +302,7 @@ RkISP1FrameInfo *RkISP1Frames::find(Request *request) int RkISP1CameraData::loadIPA() { - ipa_ = IPAManager::createIPA(pipe_, 1, 1); + ipa_ = IPAManager::createIPA(pipe_, 1, 1); if (!ipa_) return -ENOENT; @@ -311,15 +315,15 @@ int RkISP1CameraData::loadIPA() } void RkISP1CameraData::queueFrameAction(unsigned int frame, - const IPAOperationData &action) + const ipa::rkisp1::RkISP1Action &action) { - switch (action.operation) { - case RKISP1_IPA_ACTION_V4L2_SET: { - const ControlList &controls = action.controls[0]; + switch (action.op) { + case ipa::rkisp1::ActionV4L2Set: { + const ControlList &controls = action.controls; delayedCtrls_->push(controls); break; } - case RKISP1_IPA_ACTION_PARAM_FILLED: { + case ipa::rkisp1::ActionParamFilled: { PipelineHandlerRkISP1 *pipe = static_cast(pipe_); RkISP1FrameInfo *info = frameInfo_.find(frame); if (!info) @@ -336,11 +340,11 @@ void RkISP1CameraData::queueFrameAction(unsigned int frame, break; } - case RKISP1_IPA_ACTION_METADATA: - metadataReady(frame, action.controls[0]); + case ipa::rkisp1::ActionMetadata: + metadataReady(frame, action.controls); break; default: - LOG(RkISP1, Error) << "Unknown action " << action.operation; + LOG(RkISP1, Error) << "Unknown action " << action.op; break; } } @@ -612,16 +616,12 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) for (const StreamConfiguration &cfg : *config) { if (cfg.stream() == &data->mainPathStream_) { ret = mainPath_.configure(cfg, format); - streamConfig[0] = { - .pixelFormat = cfg.pixelFormat, - .size = cfg.size, - }; + streamConfig[0] = IPAStream(cfg.pixelFormat, + cfg.size); } else { ret = selfPath_.configure(cfg, format); - streamConfig[1] = { - .pixelFormat = cfg.pixelFormat, - .size = cfg.size, - }; + streamConfig[1] = IPAStream(cfg.pixelFormat, + cfg.size); } if (ret) @@ -650,12 +650,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) ret = 0; } - std::map entityControls; + std::map entityControls; entityControls.emplace(0, data->sensor_->controls()); - IPAOperationData ipaConfig; - data->ipa_->configure(sensorInfo, streamConfig, entityControls, - ipaConfig, nullptr); + data->ipa_->configure(sensorInfo, streamConfig, entityControls); return 0; } @@ -695,15 +693,15 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) for (std::unique_ptr &buffer : paramBuffers_) { buffer->setCookie(ipaBufferId++); - data->ipaBuffers_.push_back({ .id = buffer->cookie(), - .planes = buffer->planes() }); + data->ipaBuffers_.emplace_back(buffer->cookie(), + buffer->planes()); availableParamBuffers_.push(buffer.get()); } for (std::unique_ptr &buffer : statBuffers_) { buffer->setCookie(ipaBufferId++); - data->ipaBuffers_.push_back({ .id = buffer->cookie(), - .planes = buffer->planes() }); + data->ipaBuffers_.emplace_back(buffer->cookie(), + buffer->planes()); availableStatBuffers_.push(buffer.get()); } @@ -757,8 +755,7 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] ControlList *c if (ret) return ret; - IPAOperationData ipaData = {}; - ret = data->ipa_->start(ipaData, nullptr); + ret = data->ipa_->start(); if (ret) { freeBuffers(camera); LOG(RkISP1, Error) @@ -853,11 +850,12 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) if (!info) return -ENOENT; - IPAOperationData op; - op.operation = RKISP1_IPA_EVENT_QUEUE_REQUEST; - op.data = { data->frame_, info->paramBuffer->cookie() }; - op.controls = { request->controls() }; - data->ipa_->processEvent(op); + ipa::rkisp1::RkISP1Event ev; + ev.op = ipa::rkisp1::EventQueueRequest; + ev.frame = data->frame_; + ev.bufferId = info->paramBuffer->cookie(); + ev.controls = request->controls(); + data->ipa_->processEvent(ev); data->frame_++; @@ -1088,10 +1086,11 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; - IPAOperationData op; - op.operation = RKISP1_IPA_EVENT_SIGNAL_STAT_BUFFER; - op.data = { info->frame, info->statBuffer->cookie() }; - data->ipa_->processEvent(op); + ipa::rkisp1::RkISP1Event ev; + ev.op = ipa::rkisp1::EventSignalStatBuffer; + ev.frame = info->frame; + ev.bufferId = info->statBuffer->cookie(); + data->ipa_->processEvent(ev); } REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP1) diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 36325ffb..78b47916 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -34,6 +34,9 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include +#include + namespace libcamera { LOG_DEFINE_CATEGORY(VIMC) @@ -56,6 +59,8 @@ public: std::unique_ptr video_; std::unique_ptr raw_; Stream stream_; + + std::unique_ptr ipa_; }; class VimcCameraConfiguration : public CameraConfiguration @@ -311,8 +316,7 @@ int PipelineHandlerVimc::start(Camera *camera, [[maybe_unused]] ControlList *con if (ret < 0) return ret; - IPAOperationData ipaData = {}; - ret = data->ipa_->start(ipaData, nullptr); + ret = data->ipa_->start(); if (ret) { data->video_->releaseBuffers(); return ret; @@ -418,7 +422,7 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator) std::unique_ptr data = std::make_unique(this, media); - data->ipa_ = IPAManager::createIPA(this, 0, 0); + data->ipa_ = IPAManager::createIPA(this, 0, 0); if (data->ipa_ != nullptr) { std::string conf = data->ipa_->configurationFile("vimc.conf"); data->ipa_->init(IPASettings{ conf }); diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index c2604d45..84d2555f 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -96,14 +96,6 @@ LOG_DEFINE_CATEGORY(Pipeline) * when creating the camera, and shall not be modified afterwards. */ -/** - * \var CameraData::ipa_ - * \brief The IPA module used by the camera - * - * Reference to the Image Processing Algorithms (IPA) operating on the camera's - * stream(s). If no IPA exists for the camera, this field is set to nullptr. - */ - /** * \class PipelineHandler * \brief Create and manage cameras based on a set of media devices diff --git a/src/libcamera/proxy/ipa_proxy_linux.cpp b/src/libcamera/proxy/ipa_proxy_linux.cpp deleted file mode 100644 index ea6f3e5e..00000000 --- a/src/libcamera/proxy/ipa_proxy_linux.cpp +++ /dev/null @@ -1,104 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_proxy_linux.cpp - Default Image Processing Algorithm proxy for Linux - */ - -#include - -#include -#include - -#include "libcamera/internal/ipa_module.h" -#include "libcamera/internal/ipa_proxy.h" -#include "libcamera/internal/ipc_unixsocket.h" -#include "libcamera/internal/log.h" -#include "libcamera/internal/process.h" - -namespace libcamera { - -LOG_DECLARE_CATEGORY(IPAProxy) - -class IPAProxyLinux : public IPAProxy -{ -public: - IPAProxyLinux(IPAModule *ipam); - ~IPAProxyLinux(); - - int init([[maybe_unused]] const IPASettings &settings) override - { - return 0; - } - int start([[maybe_unused]] const IPAOperationData &data, - [[maybe_unused]] IPAOperationData *result) override { return 0; } - void stop() override {} - void configure([[maybe_unused]] const CameraSensorInfo &sensorInfo, - [[maybe_unused]] const std::map &streamConfig, - [[maybe_unused]] const std::map &entityControls, - [[maybe_unused]] const IPAOperationData &ipaConfig, - [[maybe_unused]] IPAOperationData *result) override {} - void mapBuffers([[maybe_unused]] const std::vector &buffers) override {} - void unmapBuffers([[maybe_unused]] const std::vector &ids) override {} - void processEvent([[maybe_unused]] const IPAOperationData &event) override {} - -private: - void readyRead(IPCUnixSocket *ipc); - - Process *proc_; - - IPCUnixSocket *socket_; -}; - -IPAProxyLinux::IPAProxyLinux(IPAModule *ipam) - : IPAProxy(ipam), proc_(nullptr), socket_(nullptr) -{ - LOG(IPAProxy, Debug) - << "initializing dummy proxy: loading IPA from " - << ipam->path(); - - std::vector fds; - std::vector args; - args.push_back(ipam->path()); - const std::string path = resolvePath("ipa_proxy_linux"); - if (path.empty()) { - LOG(IPAProxy, Error) - << "Failed to get proxy worker path"; - return; - } - - socket_ = new IPCUnixSocket(); - int fd = socket_->create(); - if (fd < 0) { - LOG(IPAProxy, Error) - << "Failed to create socket"; - return; - } - socket_->readyRead.connect(this, &IPAProxyLinux::readyRead); - args.push_back(std::to_string(fd)); - fds.push_back(fd); - - proc_ = new Process(); - int ret = proc_->start(path, args, fds); - if (ret) { - LOG(IPAProxy, Error) - << "Failed to start proxy worker process"; - return; - } - - valid_ = true; -} - -IPAProxyLinux::~IPAProxyLinux() -{ - delete proc_; - delete socket_; -} - -void IPAProxyLinux::readyRead([[maybe_unused]] IPCUnixSocket *ipc) -{ -} - -REGISTER_IPA_PROXY(IPAProxyLinux) - -} /* namespace libcamera */ diff --git a/src/libcamera/proxy/ipa_proxy_thread.cpp b/src/libcamera/proxy/ipa_proxy_thread.cpp deleted file mode 100644 index a5fda2c8..00000000 --- a/src/libcamera/proxy/ipa_proxy_thread.cpp +++ /dev/null @@ -1,175 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Google Inc. - * - * ipa_proxy_thread.cpp - Proxy running an Image Processing Algorithm in a thread - */ - -#include - -#include -#include - -#include "libcamera/internal/ipa_context_wrapper.h" -#include "libcamera/internal/ipa_module.h" -#include "libcamera/internal/ipa_proxy.h" -#include "libcamera/internal/log.h" -#include "libcamera/internal/thread.h" - -namespace libcamera { - -LOG_DECLARE_CATEGORY(IPAProxy) - -class IPAProxyThread : public IPAProxy, public Object -{ -public: - IPAProxyThread(IPAModule *ipam); - - int init(const IPASettings &settings) override; - int start(const IPAOperationData &data, - IPAOperationData *result) override; - void stop() override; - - void configure(const CameraSensorInfo &sensorInfo, - const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *result) override; - void mapBuffers(const std::vector &buffers) override; - void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPAOperationData &event) override; - -private: - void queueFrameAction(unsigned int frame, const IPAOperationData &data); - - /* Helper class to invoke processEvent() in another thread. */ - class ThreadProxy : public Object - { - public: - void setIPA(IPAInterface *ipa) - { - ipa_ = ipa; - } - - int start(const IPAOperationData &data, IPAOperationData *result) - { - return ipa_->start(data, result); - } - - void stop() - { - ipa_->stop(); - } - - void processEvent(const IPAOperationData &event) - { - ipa_->processEvent(event); - } - - private: - IPAInterface *ipa_; - }; - - bool running_; - Thread thread_; - ThreadProxy proxy_; - std::unique_ptr ipa_; -}; - -IPAProxyThread::IPAProxyThread(IPAModule *ipam) - : IPAProxy(ipam), running_(false) -{ - if (!ipam->load()) - return; - - struct ipa_context *ctx = ipam->createContext(); - if (!ctx) { - LOG(IPAProxy, Error) - << "Failed to create IPA context for " << ipam->path(); - return; - } - - ipa_ = std::make_unique(ctx); - proxy_.setIPA(ipa_.get()); - - /* - * Proxy the queueFrameAction signal to dispatch it in the caller's - * thread. - */ - ipa_->queueFrameAction.connect(this, &IPAProxyThread::queueFrameAction); - - valid_ = true; -} - -int IPAProxyThread::init(const IPASettings &settings) -{ - int ret = ipa_->init(settings); - if (ret) - return ret; - - proxy_.moveToThread(&thread_); - - return 0; -} - -int IPAProxyThread::start(const IPAOperationData &data, - IPAOperationData *result) -{ - running_ = true; - thread_.start(); - - return proxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking, - data, result); -} - -void IPAProxyThread::stop() -{ - if (!running_) - return; - - running_ = false; - - proxy_.invokeMethod(&ThreadProxy::stop, ConnectionTypeBlocking); - - thread_.exit(); - thread_.wait(); -} - -void IPAProxyThread::configure(const CameraSensorInfo &sensorInfo, - const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *result) -{ - ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, - result); -} - -void IPAProxyThread::mapBuffers(const std::vector &buffers) -{ - ipa_->mapBuffers(buffers); -} - -void IPAProxyThread::unmapBuffers(const std::vector &ids) -{ - ipa_->unmapBuffers(ids); -} - -void IPAProxyThread::processEvent(const IPAOperationData &event) -{ - if (!running_) - return; - - /* Dispatch the processEvent() call to the thread. */ - proxy_.invokeMethod(&ThreadProxy::processEvent, ConnectionTypeQueued, - event); -} - -void IPAProxyThread::queueFrameAction(unsigned int frame, const IPAOperationData &data) -{ - IPAInterface::queueFrameAction.emit(frame, data); -} - -REGISTER_IPA_PROXY(IPAProxyThread) - -} /* namespace libcamera */ diff --git a/src/libcamera/proxy/meson.build b/src/libcamera/proxy/meson.build index 5c4365e7..00ae5a8f 100644 --- a/src/libcamera/proxy/meson.build +++ b/src/libcamera/proxy/meson.build @@ -1,10 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 -libcamera_sources += files([ - 'ipa_proxy_linux.cpp', - 'ipa_proxy_thread.cpp', -]) - # generate {pipeline}_ipa_proxy.cpp foreach mojom : ipa_mojoms proxy = custom_target(mojom['name'] + '_proxy_cpp', diff --git a/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp b/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp deleted file mode 100644 index bdbac988..00000000 --- a/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp +++ /dev/null @@ -1,90 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * ipa_proxy_linux_worker.cpp - Default Image Processing Algorithm proxy worker for Linux - */ - -#include -#include -#include - -#include -#include - -#include "libcamera/internal/event_dispatcher.h" -#include "libcamera/internal/ipa_module.h" -#include "libcamera/internal/ipc_unixsocket.h" -#include "libcamera/internal/log.h" -#include "libcamera/internal/thread.h" - -using namespace libcamera; - -LOG_DEFINE_CATEGORY(IPAProxyLinuxWorker) - -void readyRead(IPCUnixSocket *ipc) -{ - IPCUnixSocket::Payload message; - int ret; - - ret = ipc->receive(&message); - if (ret) { - LOG(IPAProxyLinuxWorker, Error) - << "Receive message failed: " << ret; - return; - } - - LOG(IPAProxyLinuxWorker, Debug) << "Received a message!"; -} - -int main(int argc, char **argv) -{ - /* Uncomment this for debugging. */ -#if 0 - std::string logPath = "/tmp/libcamera.worker." + - std::to_string(getpid()) + ".log"; - logSetFile(logPath.c_str()); -#endif - - if (argc < 3) { - LOG(IPAProxyLinuxWorker, Debug) - << "Tried to start worker with no args"; - return EXIT_FAILURE; - } - - int fd = std::stoi(argv[2]); - LOG(IPAProxyLinuxWorker, Debug) - << "Starting worker for IPA module " << argv[1] - << " with IPC fd = " << fd; - - std::unique_ptr ipam = std::make_unique(argv[1]); - if (!ipam->isValid() || !ipam->load()) { - LOG(IPAProxyLinuxWorker, Error) - << "IPAModule " << argv[1] << " should be valid but isn't"; - return EXIT_FAILURE; - } - - IPCUnixSocket socket; - if (socket.bind(fd) < 0) { - LOG(IPAProxyLinuxWorker, Error) << "IPC socket binding failed"; - return EXIT_FAILURE; - } - socket.readyRead.connect(&readyRead); - - struct ipa_context *ipac = ipam->createContext(); - if (!ipac) { - LOG(IPAProxyLinuxWorker, Error) << "Failed to create IPA context"; - return EXIT_FAILURE; - } - - LOG(IPAProxyLinuxWorker, Debug) << "Proxy worker successfully started"; - - /* \todo upgrade listening loop */ - EventDispatcher *dispatcher = Thread::current()->eventDispatcher(); - while (1) - dispatcher->processEvents(); - - ipac->ops->destroy(ipac); - - return 0; -} diff --git a/src/libcamera/proxy/worker/meson.build b/src/libcamera/proxy/worker/meson.build index 0fd26fd8..3796103e 100644 --- a/src/libcamera/proxy/worker/meson.build +++ b/src/libcamera/proxy/worker/meson.build @@ -1,9 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 -ipa_proxy_sources = [ - ['ipa_proxy_linux', 'ipa_proxy_linux_worker.cpp'] -] - proxy_install_dir = join_paths(get_option('libexecdir'), 'libcamera') # generate {pipeline}_ipa_proxy_worker.cpp -- cgit v1.2.1