summaryrefslogtreecommitdiff
path: root/src/ipa/ipu3
AgeCommit message (Collapse)Author
2021-10-06ipa: ipu3: Change the limits of the AWB statsJean-Michel Hautbois
Until now, the limits used to calculate the grid based on the Bayer Down Scaler configuration where taken from the kernel documentation [0]. While testing and understanding the format of the ImgU statistics, it appears that the ones defined in CrOS [1] are the correct ones. Use those. [0] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-meta-intel-ipu3.html?highlight=v4l2_meta_fmt_ipu3_params#intel-ipu3-imgu-uapi-data-types [1] https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Make the naming consistentJean-Michel Hautbois
The variables mix the terms cell, region and zone. It can confuse the reader, and make the algorithm more difficult to follow. Rename the local variables to be consistent with their definitions: - Cells are defined in Pixels - Zones are defined in Cells There is no "region" as such, so replace it with the correct term. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Change Accumulator structure layoutJean-Michel Hautbois
The pixel component sums for the Accumulator are inconsistent with other similar structures such as the IPAFrameContext::awb::gains. Group the red, green, and blue sums together in a struct and store them as uint64_t to reduce potential architectural differences. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Rename IspStatsRegion to AccumulatorJean-Michel Hautbois
The IspStatsRegion structure was introduced as an attempt to prepare for a generic AWB algorithm structure. The structure name by itself is not explicit and it is too optimistic to try and make a generic one for now. Its role is to accumulate the pixels in a given zone. Rename it to accumulator, and remove the uncounted field at the same time. It is always possible to know how many pixels are not relevant for the algorithm by calculating total-counted. The uncounted field was only declared and not used. Amend the documentation accordingly. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Move the AWB stats structuresJean-Michel Hautbois
The structure Ipu3AwbCell describes the AWB stats layout on the kernel side. We will need it to be used by the AGC algorithm to be introduced later, so let's make it visible from ipa::ipu3::algorithms and not only for the AWB class. The IspStatsRegion will be needed by AGC too, so let's move it in the same namespace too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-09-27libcamera: ipu3: Drop entityControls mapJacopo Mondi
The IPA::configure() function has an IPAConfigInfo parameters which contains a map of numerical indexes to ControlInfoMap instances. This is a leftover of the old IPA protocol, where it was not possible to specify a rich interface as it is possible today and each entity ControlInfoMap was indexed by a numerical id and stored in a map. Now that the IPA interface allows to specify parameters by name, drop the map and send the sensor's control info map only. If we'll need more ControlInfoMap to be shared with the IPA, a new parameter can be added to IPAConfigInfo. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-09-14ipa: ipu3: Clear incoming parameter use flagsKieran Bingham
The incoming params buffer may contain uninitialised data, or the parameters of previously queued frames. Clearing the entire buffer may be an expensive operation, and the kernel will only read from structures which have their associated use-flag set. It is the responsibility of the algorithms to set the use flags accordingly for any data structure they update during prepare(). Clear the use flags of the parameter buffer before passing the buffer to the algorithms during their prepare() operations. Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31libcamera: mapped_framebuffer: Rename maps() to planes()Hirokazu Honda
MappedFrameBuffer::maps() returns planes_. This renames the function name to planes(). Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-23ipa: ipu3: awb: remove local storage of the gridJean-Michel Hautbois
The IPASessionConfiguration now has the grid configuration stored. Use it it at prepare() and process() calls in AWB and pass it as a reference to the private functions when needed. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-23ipa: ipu3: agc: remove local storage of the gridJean-Michel Hautbois
The IPASessionConfiguration now has the grid configuration stored. Use it at process() call in AGC and pass it as a reference to the private functions when needed. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-20ipa: ipu3: Move IPU3 agc into algorithmsJean-Michel Hautbois
Now that the interface is properly used by the AGC class, move it into ipa::ipu3::algorithms and let the loops do the calls. As we need to exchange the exposure_ and gain_ by passing them through the FrameContext, use the calculated values in setControls() function to ease the reading. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Move IPU3 awb into algorithmsJean-Michel Hautbois
Now that the interface is properly used by the AWB class, move it into ipa::ipu3::algorithms and let the loops do the calls. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: convert AGC to the new algorithm interfaceJean-Michel Hautbois
In preparation for using the AGC through the new algorithm interfaces, convert the existing code to use the new function types. Now that the process call is rewritten, re-enable the compiler flag to warn when a function declaration hides virtual functions from a base class (-Woverloaded-virtual). We never use converged_ so remove its declaration. The controls may not need to be updated at each call, but it should be decided on the context side and not by a specific call by using a lock status in the Agc structure for instance. As the params_ local variable is not useful anymore, remove it here too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: convert AWB to the new algorithm interfaceJean-Michel Hautbois
When the stats are received, pass them with the context to the existing AWB algorithm. IPAFrameContext now has a new structure to store the gains calculated by the AWB algorithm. When an EventFillParams event is received, call prepare() and set the new gains accordingly in the params structure. There is no more a need for the IPU3Awb::initialise() function, as the params are always set in prepare(). Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce a modular tone mapping algorithmJean-Michel Hautbois
Introduce a new algorithm to manage the tone mapping handling of the IPU3. The initial algorithm is chosen to configure the gamma contrast curve which moves the implementation out of AWB for simplicity. As it is initialised with a default gamma value of 1.1, there is no need to use the default table at initialisation anymore. This demonstrates the way to use process() call when the EventStatReady comes in. The function calculates the LUT in the context of a frame, and when prepare() is called, the parameters are filled with the updated values. AGC is modified to take the new process interface into account. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce modular algorithmJean-Michel Hautbois
Implement a new modular framework for algorithms with a common context structure that is passed to each algorithm through a common API. This patch: - removes all the local references from IPAIPU3 and uses IPAContext - implements the list of pointers and the loop at configure call on each algorithm - loops in fillParams on each prepare() call on the algorithm list - loops in prepareStats on each process() call on the algorithm list Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Add the functions to the Algorithm classJean-Michel Hautbois
Introduce three functions in the Algorithm class to manage algorithms: - configure which is called when IPA is configured only - prepare called on EventFillParams event at each frame when the request is queued - process called on EventStatReady event at each frame completion when the statistics have been generated. The existing AGC implementation already has a function named process(), though it has different arguments. Adding the new virtual process() interface causes a compiler warning due to the AGC implementation overloading a virtual function, even though the overload can be resolved correctly. Temporarily disable the warning in this commit to maintain bisection until the AGC is converted to the new interface. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce a Context structureJean-Michel Hautbois
An increasing amount of data and information needs to be shared between the components that build up to implement image processing algorithms. Create a context structure which will allow us to work towards calling algorithms in a modular way, and sharing information between the modules. The IPA context is a global context set at configure time (IPASessionConfiguration) and a per-frame context (IPAFrameContext) used while streaming. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: move libipa::Algorithm to ipa/ipu3/algorithmsJean-Michel Hautbois
The abstract Algorithm class was originally placed in libipa as an attempt define a generic algorithm container. This was a little optimistic and pushed a bit far too early. Move the Algorithm class into the IPU3 which is the only user of the class, as we adapt it to support modular algorithm components for the IPU3. Not documenting the namespace may cause issues with Doxygen in libipa. The file libipa.cpp is thus created as an empty file for now, but we can leverage it in the future to add more global libipa documentation, and possibly code too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-12ipa: ipu3: Tidy-up includesJacopo Mondi
Tidy-up a bit the inclusions directive in the IPU3 IPA module. In detail: - ipu3.cpp is missing inclusions for: std::abs from <cmath> std::map from <map> std::min/max from <algorithm> std::numeric_limits from <limits> std::unique_ptr from <memory> std::vector from <vector> and does not require <sys/mman.h> - ipu3_agc has two not used inclusions in the header file and one the cpp file and is missing <chrono> for std::literals::chrono_literals - ipu3_awb is missing <algorithm> for std::sort and does not use <numeric> or <unordered_map> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-08-12libcamera: ipu3: Initialize controls in the IPAJacopo Mondi
All the IPU3 Camera controls are currently initialized by the pipeline handler which initializes them using the camera sensor configuration and platform specific requirements. However, some controls are better initialized by the IPA, which might, in example, cap the exposure times and frame duration to the constraints of its algorithms implementation. Also, moving forward, the IPA should register controls to report its capabilities, in example the ability to enable/disable 3A algorithms on request. Move the existing controls initialization to the IPA, by providing the sensor configuration and its controls to the IPU3IPA::init() function, which initializes controls and returns them to the pipeline through an output parameter. The existing controls initialization has been copied verbatim from the pipeline handler to the IPA, if not a for few line breaks adjustments and the resulting Camera controls values are not changed. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-08-10libcamera: MappedFrameBuffer: Use typed Flags<MapModes>Kieran Bingham
Remove the need for callers to reference PROT_READ/PROT_WRITE directly from <sys/mman.h> by instead exposing the Read/Write mapping options as flags from the MappedFrameBuffer class itself. While here, introduce the <stdint.h> header which is required for the uint8_t as part of the Plane. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-10libcamera: Give MappedFrameBuffer its own implementationKieran Bingham
The MappedFrameBuffer is a convenience feature which sits on top of the FrameBuffer and facilitates mapping it to CPU accessible memory with mmap. This implementation is internal and currently sits in the same internal files as the internal FrameBuffer, thus exposing those internals to users of the MappedFramebuffer implementation. Move the MappedFrameBuffer and MappedBuffer implementation to its own implementation files, and fix the sources throughout to use that accordingly. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-09libcamera: Rename 'method' to 'function'Laurent Pinchart
Usage of 'method' to refer to member functions comes from Java. The C++ standard uses the term 'function' only. Replace 'method' with 'function' or 'member function' through the whole code base and documentation. While at it, fix two typos (s/backeng/backend/). The BoundMethod and Object::invokeMethod() are left as-is here, and will be addressed separately. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-07-11libcamera: buffer: Rename buffer.h to framebuffer.hLaurent Pinchart
libcamera names header files based on the classes they define. The buffer.h file is an exception. Rename it to framebuffer.h. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-28ipa: ipu3: Initialize CameraSensorHelper at IPU3 init stageJean-Michel Hautbois
In order for the CameraSensorHelper to be instantiated, we need to find its factory using the camera sensor model name stored in IPASettings::sensorModel. As we don't need to do it at each configure call (the sensor is not changing in-between), implement the init call in IPAIPU3 to do that. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Validate internal headers as privateKieran Bingham
Headers which must not be exposed as part of the public libcamera API should include base/private.h. Any interface which includes the private.h header will only be able to build if the libcamera_private dependency is used (or the libcamera_base_private dependency directly). Build targets which are intended to use the private API's will use the libcamera_private to handle the automatic definition of the inclusion guard. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Move extended base functionalityKieran Bingham
Move the functionality for the following components to the new base support library: - BoundMethod - EventDispatcher - EventDispatcherPoll - Log - Message - Object - Signal - Semaphore - Thread - Timer While it would be preferable to see these split to move one component per commit, these components are all interdependent upon each other, which leaves us with one big change performing the move for all of them. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Move utils to the base libraryKieran Bingham
Move the utils functionality to the libcamera/base library. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-24ipa: ipu3: Use libcamera::utils::Duration helper class for durationsUmang Jain
std::chrono::Duration is provided quite conveniently by libcamera::utils::Duration wrapper. Port IPAIPU3 to use that for duration-type entities (such as exposure time), such that it becomes consistent with rest of the codebase. The commit doesn't introduce any functional changes. Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2021-06-14ipa: ipu3: Support return values from configure()Kieran Bingham
The IPU3 IPA interface does not define a return value from configure(). This prevents errors from being reported back to the pipeline handler when they occur in the IPA. Update the IPU3 IPA interface and add return values to the checks in IPAIPU3::configure() accordingly Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-09ipa: ipu3: Fix struct/class mismatch in forward declarationLaurent Pinchart
The ipu3_agc.h forward-declares the IPACameraSensorInfo structure, but incorrectly declares it as a class. This causes a compilation error with clang: include/libcamera/ipa/core_ipa_interface.h:24:1: error: 'IPACameraSensorInfo' defined as a struct here but previously declared as a class; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Werror,-Wmismatched-tags] struct IPACameraSensorInfo ^ ../../src/ipa/ipu3/ipu3_agc.h:21:1: note: did you mean struct here? class IPACameraSensorInfo; ^~~~~ struct Fix it. Fixes: 384a53d3cdf7 ("ipa: ipu3: Calculate line duration from IPACameraSensorInfo") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2021-06-08ipa: ipu3: Calculate frame duration from minimum VBLANK valueUmang Jain
Frame duration is hard-coded for CTS as per [1]. Ideally, to accurately calculate the frame duration, it needs the VBLANK value from every frame's exposure. However, this particular bit is yet to be implemented in IPAIPU3. Meanwhile, we can at least head in the right direction by not hard coding the value, instead using the default VBLANK value as reported by the sensor. Update the existing \todo, to use the derived VBLANK value as and when it's available from each frame exposure. [1] 6c5f3fe6ced7 ("ipa: ipu3: Set output frame duration metadata") Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-06-08ipa: ipu3: Copy IPACameraSensorInfo for future usageUmang Jain
IPACameraSensorInfo members will be needed at various places in the IPAIPU3 class, in subsequent commits. Hence, it seems trivial to copy this structure for wider availability throughout the class. This commit does not introduce any functional changes. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-06-08ipa: ipu3: Calculate line duration from IPACameraSensorInfoUmang Jain
Squash \todo by calculating line duration from IPACameraSensorInfo, now passed in, to IPU3Agc::initialise(). Since line duration is now calculated from real values, store it as a private member in IPU3Agc class. As a further step, replace the associated global constant, kMaxExposureTime, with a private IPU3Agc class member as well, and assign its value correspondingly in IPU3Agc::initialise(), similar to previous precedence. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-06-01ipa: ipu3: Use a default gamma LUT at startupJean-Michel Hautbois
A custom gamma LUT is used, but the table is not calculated at startup, resulting in an all 0's frame. Use the drfault LUT specified in the Linux kernel as a first estimation. Reported-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-05-27ipa: ipu3: Set output frame duration metadataPaul Elder
The sensor frame duration should be set by the IPA. Hardcode it for now. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-05-26ipa: ipu3: Provide frame timestamps through IPU3EventUmang Jain
Pass in frame timestamps from IPU3 pipeline handler to IPU3 IPA via IPU3Event. Frame timestamps are helpful to IPA algorithms to convergence, by setting them via IPA stats. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-05-24ipa: ipu3: Introduce IPAConfigInfo in IPCUmang Jain
IPAConfigInfo is a consolidated data structure passed from IPU3 pipeline-handler to IPU3 IPA. The structure can be extended with additional parameters to accommodate the requirements of multiple IPU3 IPA modules. Adapt the in-tree IPU3 IPA to use IPAConfigInfo as well. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-05-24ipa: ipc: Rename CameraSensorInfo to IPACameraSensorInfoUmang Jain
This matches the naming convention for IPA IPC. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Acked-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-04-22ipa: ipu3: fix coverity issues in AWBJean-Michel Hautbois
The BNR configuration structure has inconsistent initialisation of two members. This was picked up by Coverity, after merge. While this code is still functional, fix them to be consistent with the other initialisations within the struct. Reported-by: Coverity CID=320056 Reported-by: Coverity CID=320058 Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-04-22ipa: ipu3: Add support for IPU3 AEC/AGC algorithmJean-Michel Hautbois
Implement basic auto-exposure (AE) and auto-gain (AG) correction functions. The functions computeTargetExposure() and computeGain() are adapted from the Raspberry Pi AGC implementation to suit the IPU3 structures, and filtering is added to reduce visible stepsize when there are large exposure changes. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-04-22ipa: ipu3: Add support for IPU3 AWB algorithmJean-Michel Hautbois
The IPA will locally modify the parameters before they are passed down to the ImgU. Use a local parameter object to give a reference to those algorithms. Inherit from the Algorithm class to implement basic AWB functions. The configure() call will set exposure and gain to their minimum value, so while AGC is not there, the frames will be dark. Once AWB is done, a color temperature is estimated and a default CCM matrix will be used (yet to be tuned). Implement a basic "grey-world" AWB algorithm just for demonstration purpose. The BDS output size is passed by the pipeline handler to the IPA. The best grid is then calculated to maximize the number of pixels taken into account in each cells. As commented in the source code, it can be improved, as it has (at least) one limitation: if a cell is big (say 128 pixels wide) and indicated as saturated, it won't be taken into account at all. Maybe is it possible to have a smaller one, at the cost of a few pixels to lose, in which case we can center the grid using the x_start and y_start parameters. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-04-17ipa: ipu3: Move the IPA to the ipa::ipu3 namespaceKieran Bingham
Simplify name-spacing of the IPU3 components by placing it in the ipa::ipu3 namespace directly. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-03-24ipa: ipu3: Do not set controls during configureKieran Bingham
The configure operation is synchronous and should not send events back to the pipeline handler. If information needs to be returned from configure it should be handled through the interface directly. Move the initial call to setControls() out of configure() and into the start() method which is called after the IPA running_ state is updated. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-03-17libcamera: ipu3: Pass the BDS rectangle at IPA configure callJean-Michel Hautbois
The IPU3 IPA will need the BDS configuration when the AWB/AGC algorithm will be integrated. In order to do that, the configure() interface needs to be modified. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-02-19ipa: ipu3: Don't call SetControls for each frameJean-Michel Hautbois
Setting a default EXPOSURE and ANALOGUE_GAIN makes sense for the first frame but there is no need to force it for all frames. It will be called later by the 3A algorithms. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-02-16libcamera: IPAInterface: Replace C API with the new C++-only APIPaul Elder
Remove everything related to the C API, including ipa_context, ipa_context_wrapper, and IPAInterfaceWrapper. Also remove relevant documentation. ipaCreate() provided by IPA implementations, and createInterface() provided by IPAModule (wrapper around IPA implementation) both now return a C++ object IPAInterface instead of struct ipa_context. Although IPAInterfaceWrapper is the only component of libipa, the skeleton and build files for libipa are retained. After converting the C API to the C++-only API, make all pipeline handlers and IPAs use the new API. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- This is a combination of 21 commits: --- libcamera: IPAModule: Replace ipa_context with IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate_() and createInterface() return IPAInterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: ipa_context_wrapper: Remove ipa_context_wrapper Since ipa_context has been replaced with custom IPAInterfaces, it is not longer needed. Remove it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAInterface: remove ipa_context and functions from documentation Remove all the documentation related to ipa_context and the C IPA API, as well as the documentation about the functions in the IPAInterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> --- libcamera: IPAInterface: Remove all functions from IPAInterface Now that all the functions in the IPA interface are defined in the data definition file and a specialized IPAInterface is generated per pipeline handler, remove all the functions from the base IPAInterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAInterface: make ipaCreate return IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate return IPAinterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> --- ipa: remove IPAInterfaceWrapper As every pipeline has its own proxy, IPAInterfaceWrapper is no longer necessary. Remove it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Acked-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> --- libcamera: IPAProxy: Remove stop() override Since stop() is part of the IPA interface, and the IPA interface is now generated based on the data definition file per pipeline, this no longer needs to be overrided by the base IPAProxy. Remove it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAProxy, IPAManager: Switch to one-proxy-per-pipeline scheme IPAProxy is changed in two major ways: - Every pipeline has its own proxy, to support each pipeline's IPA interface - IPAProxy implementations always encapsulate IPA modules, and switch internally for isolation or threaded The IPAProxy registration mechanism is removed, as each pipeline will have its own proxy, so the pipeline can pass the specialized class name of the IPAProxy to the IPAManager for construction. IPAManager is changed accordingly to support these changes: - createIPA is a template function that takes an IPAProxy class, and always returns an IPAProxy - IPAManager no longer decides on isolation, and simply creates an IPAProxy instance while passing the isolation flag Consequently, the old IPAProxy classes (IPAProxyThread and IPAProxyLinux) are removed. The IPAInterfaceTest is updated to use the new IPAManager interface, and to construct a ProcessManager as no single global instance is created anymore. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> --- libcamera: IPAProxy: Add isolate parameter to create() Since IPAProxy implementations now always encapsulate IPA modules, add a parameter to create() to signal if the proxy should isolate the IPA or not. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline Now that each pipeline handler has its own IPAProxy implementation, make the IPAManager fetch the IPAProxy based on the pipeline handler name. Also, since the IPAProxy is used regardless of isolation or no isolation, remove the isolation check from the proxy selection. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> --- libcamera: IPAManager: add isolation flag to proxy creation When the IPA proxy is created, it needs to know whether to isolate or not. Feed the flag at creation of the IPA proxy. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAManager: Make createIPA return proxy directly Since every pipeline knows the type of the proxy that it needs, and since all IPAs are to be wrapped in a proxy, IPAManager no longer needs to search in the factory list to fetch the proxy factory to construct a factory. Instead, we define createIPA as a template function, and the pipeline handler can declare the proxy type when it calls createIPA. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAProxy: Remove registration mechanism Implementations of IPA proxies use a registration mechanism to register themselves with the main IPA proxy factory. This registration declares static objects, causing a risk of things being constructed before the proper libcamera facilities are ready. Since each pipeline handler has its own IPA proxy and knows the type, it isn't necessary to have a proxy factory. Remove it to alleviate the risk of early construction. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread We have now changed the proxy from per-IPC mechanism to per-pipeline. The per-IPC mechanism proxies are thus no longer needed; remove them. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- tests: ipa_interface_test: Update to use new createIPA Update the IPA interface test to use the new createIPA function from IPAManager. Also create an instance of ProcessManager, as no single global instance is created automatically anymore. Update meson.build to to depend on the generated IPA interface headers. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: PipelineHandler: Remove IPA from base class Since pipeline handlers now have their own IPA interface types, it can no longer be defined in the base class, and each pipeline handler implementation must declare it and its type themselves. Remove it from the base class. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- ipa: raspberrypi: Add mojom data definition file Add a mojom data definition for raspberrypi pipeline handler's IPAs. This simplifies the API between the raspberrypi pipeline handler and the IPA, and is not a direct translation of what was used before with IPAOperationData. Also move the enums from raspberrypi.h to raspberrypi.mojom Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: pipeline, ipa: raspberrypi: Use new data definition Now that we can generate custom functions and data structures with mojo, switch the raspberrypi pipeline handler and IPA to use the custom data structures as defined in the mojom data definition file. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: pipeline, ipa: vimc: Support the new IPC mechanism Add support to vimc pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> --- libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism Add support to the rkisp1 pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: pipeline, ipa: ipu3: Support the new IPC mechanism Add support to ipu3 pipeline handler and IPA for the new IPC mechanism. [Original version] Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> [Fixed commit message and small changes] Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-02-11meson: Fix coding style when declaring arraysLaurent Pinchart
The meson.build files mix array declarations with and without a space after the opening and before the closing square bracket. The vast majority of cases don't use spaces, so standardize on that. While it it, fix indentation in a few places. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-02-05libcamera: ipa: ipu3: Add an IPA skeleton for the IPU3 pipelineNiklas Söderlund
Add an empty IPA skeleton for the IPU3 pipeline. The skeleton IPA handles the flow of parameter and statistic buffers but does not read or write anything in the buffers. It also allows the IPA to set sensor controls but does not implement any logic to set optimal values and instead sets the V4L2 exposure and gain controls to max and keeps them at that setting. This IPA is meant as a base to allow the pipeline handler to be wired up to an IPA. The image algorithms can then later be added to the IPA independently from also having to add plumbing to the pipeline handler. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>