summaryrefslogtreecommitdiff
path: root/src/ipa/ipu3/ipu3.cpp
AgeCommit message (Collapse)Author
2021-11-15ipa: ipu3: Send color temperature in the metadataJean-Michel Hautbois
Now that the color temperature is updated per-frame, use the value and set the corresponding controls::ColourTemperature. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Use sensor controls to update frameContextJean-Michel Hautbois
The pipeline handler populates the new sensorControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Make those values the frameContext::sensor values for the frame when the EventStatReady event is received. AGC also needs to use frameContext.sensor as its input values and frameContext.agc as its output values. Modify computeExposure by passing it the frameContext instead of individual exposure and gain values. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Extend ipu3 ipa interface for sensor controlsHan-Lin Chen
IPU3Event and IPU3Action use single ControlList for both libcamera and V4L2 controls, and it's content could be either one based on the context. Extend IPU3Event and IPU3Action for sensor V4L2 controls, and preserve the original one for only libcamera Controls to make the content of an event more specific. Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org> [Jean-Michel: remove lensControls from the original patch] Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-06ipa: ipu3: List the events in their order of callingJean-Michel Hautbois
The IPU3 IPA has three events which are handled from the pipeline handler. The events are received in the sequence, EventProcessControls, EventFillParams, and finally EventStatReady, while the code lists these in a different order. Update the flow of IPAIPU3::processEvent() to match the expected sequence of events, to help support the reader in interpreting the flow of events through the IPA. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-10-26ipa: ipu3: Implement an empty stop() functionJean-Michel Hautbois
While the stop() function does not currently perform any action, it forms part of the IPA interface and is a public function in the class. Promote it to a full (but basic) function implementation and begin the documentation accordingly so that there is an appropriate stub to perform stop operations if they come up. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26ipa: ipu3: Isolate ipa_context documentationKieran Bingham
The IPU3 IPA core is growing with additional documentation. The ipa_context documentation is stored here, but it pushes the IPU3 documentation and implementation further from the head of the file. Furthermore, the ipa_context documentation is outside of the ipa::ipu3 namespace and isn't identified correctly by Doxygen. Move the ipa_context to its own compilation object even though there isn't any code, but to maintain consistency with our documentation model. Correctly re-introduce the documentation into the libcamera::ipa::ipu3 namespace during the move. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26ipa: ipu3: tonemapping: Generate the LUT only on gamma changeJean-Michel Hautbois
The tone mapping algorithm calculates the gamma curve for every frame, regardless of whether the gamma value has changed or not. This issue is exasperated as we currently hardcode the gamma to a single value. Optimise the implementation to only recalculate the look up table when the gamma setting is changed, and store the gamma setting of the LUT curve as part of the IPA context. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26ipa: ipu3: Explicitly use the statistics parameterJean-Michel Hautbois
The stats pointer is marked as [[maybe_unused]]. This is a leftover from a previous commit which was here to keep the compatibility while transitioning to the new iterative algorithms. Remove this attribute to make it explicit that stats are really used to feed the algorithms. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26ipa: ipu3: Document the IPAIPU3 classJean-Michel Hautbois
Clarify the roles and interactions between the pipeline handler events and the algorithm calls by documenting all the remaining functions of the class. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26ipa: ipu3: Document IPAIPU3::configureJean-Michel Hautbois
Further extend the documentation for the IPAIPU3::configure operation. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26ipa: ipu3: Document IPAIPU3 class interfaceJean-Michel Hautbois
The IPU3 IPA is maturing to a modular and extensible system capable of handling specific algorithms for the processing blocks on the ImgU. Provide a top-level class documentation to provide an overview of the IPA, detailing what events are used and what algorithms are currently supported, as well as the limitations currently imposed. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-22ipa: ipu3: set frameContext before controlsJean-Michel Hautbois
The AGC frame context needs to be initialised correctly for the first iteration. Until now, the IPA uses the minimum exposure and gain values and caches those in local variables. In order to give the sensor limits to AGC, create a new structure in IPASessionConfiguration. Store the exposure in time (and not line duration) and the analogue gain after CameraSensorHelper conversion. Set the gain and exposure appropriately to the current values known to the IPA and remove the setting of exposure and gain in IPAIPU3 as those are now fully controlled by IPU3Agc. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15ipa: ipu3: Update camera controls in configure()Jacopo Mondi
When a new CameraConfiguration is applied to the Camera the IPA is configured as well, using the newly applied sensor configuration and its updated V4L2 controls. Also update the Camera controls at IPA::configure() time by re-computing the controls::ExposureTime and controls::FrameDurationLimits limits and update the controls on the pipeline handler side after having configured the IPA. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Introduce Black Level CorrectionJean-Michel Hautbois
The pixels output by the camera normally include a black level, because sensors do not always report a signal level of '0' for black. Pixels at or below this level should be considered black and to achieve that, we need to substract an offset to all the pixels. This can be taken into account by reading the lowest value of a special region on sensors which is not exposed to light. This provides a substracting factor to be able to adjust the expected black levels in the resulting images. For a camera outputting 10-bit pixel values (in the range 0 to 1023) a typical black level might be 64. It is a fixed value, obtained by capturing a raw frame with minimum exposure and gain fixed to 1.0 while covering the sensor (the darker the better). We consider it good enough as a very first approximation, until we measure it during a tuning process and include it in a configuration file Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Use the line stride for the statsJean-Michel Hautbois
The statistics buffer 'ipu3_uapi_awb_raw_buffer' stores the ImgU calculation results in a buffer aligned horizontally to a multiple of 4 cells. The AWB loop should take care of it to add the proper offset between lines and avoid any staircase effect. It is no longer required to pass the grid configuration context to the private functions called from process() which simplifies the code flow. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Change limits and split loops in calculateBdsGrid()Jean-Michel Hautbois
The loops over the width and height of the image when calculating the BDS grid parameters are nested, but they're actually independent. Split them to reduce the complexity. While at it, split out the constants to documented const expressions for the grid sizes. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Change the limits of the AWB statsJean-Michel Hautbois
Until now, the limits used to calculate the grid based on the Bayer Down Scaler configuration where taken from the kernel documentation [0]. While testing and understanding the format of the ImgU statistics, it appears that the ones defined in CrOS [1] are the correct ones. Use those. [0] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-meta-intel-ipu3.html?highlight=v4l2_meta_fmt_ipu3_params#intel-ipu3-imgu-uapi-data-types [1] https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-09-27libcamera: ipu3: Drop entityControls mapJacopo Mondi
The IPA::configure() function has an IPAConfigInfo parameters which contains a map of numerical indexes to ControlInfoMap instances. This is a leftover of the old IPA protocol, where it was not possible to specify a rich interface as it is possible today and each entity ControlInfoMap was indexed by a numerical id and stored in a map. Now that the IPA interface allows to specify parameters by name, drop the map and send the sensor's control info map only. If we'll need more ControlInfoMap to be shared with the IPA, a new parameter can be added to IPAConfigInfo. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-09-14ipa: ipu3: Clear incoming parameter use flagsKieran Bingham
The incoming params buffer may contain uninitialised data, or the parameters of previously queued frames. Clearing the entire buffer may be an expensive operation, and the kernel will only read from structures which have their associated use-flag set. It is the responsibility of the algorithms to set the use flags accordingly for any data structure they update during prepare(). Clear the use flags of the parameter buffer before passing the buffer to the algorithms during their prepare() operations. Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31libcamera: mapped_framebuffer: Rename maps() to planes()Hirokazu Honda
MappedFrameBuffer::maps() returns planes_. This renames the function name to planes(). Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Move IPU3 agc into algorithmsJean-Michel Hautbois
Now that the interface is properly used by the AGC class, move it into ipa::ipu3::algorithms and let the loops do the calls. As we need to exchange the exposure_ and gain_ by passing them through the FrameContext, use the calculated values in setControls() function to ease the reading. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Move IPU3 awb into algorithmsJean-Michel Hautbois
Now that the interface is properly used by the AWB class, move it into ipa::ipu3::algorithms and let the loops do the calls. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: convert AGC to the new algorithm interfaceJean-Michel Hautbois
In preparation for using the AGC through the new algorithm interfaces, convert the existing code to use the new function types. Now that the process call is rewritten, re-enable the compiler flag to warn when a function declaration hides virtual functions from a base class (-Woverloaded-virtual). We never use converged_ so remove its declaration. The controls may not need to be updated at each call, but it should be decided on the context side and not by a specific call by using a lock status in the Agc structure for instance. As the params_ local variable is not useful anymore, remove it here too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: convert AWB to the new algorithm interfaceJean-Michel Hautbois
When the stats are received, pass them with the context to the existing AWB algorithm. IPAFrameContext now has a new structure to store the gains calculated by the AWB algorithm. When an EventFillParams event is received, call prepare() and set the new gains accordingly in the params structure. There is no more a need for the IPU3Awb::initialise() function, as the params are always set in prepare(). Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce a modular tone mapping algorithmJean-Michel Hautbois
Introduce a new algorithm to manage the tone mapping handling of the IPU3. The initial algorithm is chosen to configure the gamma contrast curve which moves the implementation out of AWB for simplicity. As it is initialised with a default gamma value of 1.1, there is no need to use the default table at initialisation anymore. This demonstrates the way to use process() call when the EventStatReady comes in. The function calculates the LUT in the context of a frame, and when prepare() is called, the parameters are filled with the updated values. AGC is modified to take the new process interface into account. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce modular algorithmJean-Michel Hautbois
Implement a new modular framework for algorithms with a common context structure that is passed to each algorithm through a common API. This patch: - removes all the local references from IPAIPU3 and uses IPAContext - implements the list of pointers and the loop at configure call on each algorithm - loops in fillParams on each prepare() call on the algorithm list - loops in prepareStats on each process() call on the algorithm list Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce a Context structureJean-Michel Hautbois
An increasing amount of data and information needs to be shared between the components that build up to implement image processing algorithms. Create a context structure which will allow us to work towards calling algorithms in a modular way, and sharing information between the modules. The IPA context is a global context set at configure time (IPASessionConfiguration) and a per-frame context (IPAFrameContext) used while streaming. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-12ipa: ipu3: Tidy-up includesJacopo Mondi
Tidy-up a bit the inclusions directive in the IPU3 IPA module. In detail: - ipu3.cpp is missing inclusions for: std::abs from <cmath> std::map from <map> std::min/max from <algorithm> std::numeric_limits from <limits> std::unique_ptr from <memory> std::vector from <vector> and does not require <sys/mman.h> - ipu3_agc has two not used inclusions in the header file and one the cpp file and is missing <chrono> for std::literals::chrono_literals - ipu3_awb is missing <algorithm> for std::sort and does not use <numeric> or <unordered_map> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-08-12libcamera: ipu3: Initialize controls in the IPAJacopo Mondi
All the IPU3 Camera controls are currently initialized by the pipeline handler which initializes them using the camera sensor configuration and platform specific requirements. However, some controls are better initialized by the IPA, which might, in example, cap the exposure times and frame duration to the constraints of its algorithms implementation. Also, moving forward, the IPA should register controls to report its capabilities, in example the ability to enable/disable 3A algorithms on request. Move the existing controls initialization to the IPA, by providing the sensor configuration and its controls to the IPU3IPA::init() function, which initializes controls and returns them to the pipeline through an output parameter. The existing controls initialization has been copied verbatim from the pipeline handler to the IPA, if not a for few line breaks adjustments and the resulting Camera controls values are not changed. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-08-10libcamera: MappedFrameBuffer: Use typed Flags<MapModes>Kieran Bingham
Remove the need for callers to reference PROT_READ/PROT_WRITE directly from <sys/mman.h> by instead exposing the Read/Write mapping options as flags from the MappedFrameBuffer class itself. While here, introduce the <stdint.h> header which is required for the uint8_t as part of the Plane. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-10libcamera: Give MappedFrameBuffer its own implementationKieran Bingham
The MappedFrameBuffer is a convenience feature which sits on top of the FrameBuffer and facilitates mapping it to CPU accessible memory with mmap. This implementation is internal and currently sits in the same internal files as the internal FrameBuffer, thus exposing those internals to users of the MappedFramebuffer implementation. Move the MappedFrameBuffer and MappedBuffer implementation to its own implementation files, and fix the sources throughout to use that accordingly. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-09libcamera: Rename 'method' to 'function'Laurent Pinchart
Usage of 'method' to refer to member functions comes from Java. The C++ standard uses the term 'function' only. Replace 'method' with 'function' or 'member function' through the whole code base and documentation. While at it, fix two typos (s/backeng/backend/). The BoundMethod and Object::invokeMethod() are left as-is here, and will be addressed separately. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-07-11libcamera: buffer: Rename buffer.h to framebuffer.hLaurent Pinchart
libcamera names header files based on the classes they define. The buffer.h file is an exception. Rename it to framebuffer.h. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-28ipa: ipu3: Initialize CameraSensorHelper at IPU3 init stageJean-Michel Hautbois
In order for the CameraSensorHelper to be instantiated, we need to find its factory using the camera sensor model name stored in IPASettings::sensorModel. As we don't need to do it at each configure call (the sensor is not changing in-between), implement the init call in IPAIPU3 to do that. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Move extended base functionalityKieran Bingham
Move the functionality for the following components to the new base support library: - BoundMethod - EventDispatcher - EventDispatcherPoll - Log - Message - Object - Signal - Semaphore - Thread - Timer While it would be preferable to see these split to move one component per commit, these components are all interdependent upon each other, which leaves us with one big change performing the move for all of them. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-14ipa: ipu3: Support return values from configure()Kieran Bingham
The IPU3 IPA interface does not define a return value from configure(). This prevents errors from being reported back to the pipeline handler when they occur in the IPA. Update the IPU3 IPA interface and add return values to the checks in IPAIPU3::configure() accordingly Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-08ipa: ipu3: Calculate frame duration from minimum VBLANK valueUmang Jain
Frame duration is hard-coded for CTS as per [1]. Ideally, to accurately calculate the frame duration, it needs the VBLANK value from every frame's exposure. However, this particular bit is yet to be implemented in IPAIPU3. Meanwhile, we can at least head in the right direction by not hard coding the value, instead using the default VBLANK value as reported by the sensor. Update the existing \todo, to use the derived VBLANK value as and when it's available from each frame exposure. [1] 6c5f3fe6ced7 ("ipa: ipu3: Set output frame duration metadata") Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-06-08ipa: ipu3: Copy IPACameraSensorInfo for future usageUmang Jain
IPACameraSensorInfo members will be needed at various places in the IPAIPU3 class, in subsequent commits. Hence, it seems trivial to copy this structure for wider availability throughout the class. This commit does not introduce any functional changes. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-06-08ipa: ipu3: Calculate line duration from IPACameraSensorInfoUmang Jain
Squash \todo by calculating line duration from IPACameraSensorInfo, now passed in, to IPU3Agc::initialise(). Since line duration is now calculated from real values, store it as a private member in IPU3Agc class. As a further step, replace the associated global constant, kMaxExposureTime, with a private IPU3Agc class member as well, and assign its value correspondingly in IPU3Agc::initialise(), similar to previous precedence. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-05-27ipa: ipu3: Set output frame duration metadataPaul Elder
The sensor frame duration should be set by the IPA. Hardcode it for now. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-05-26ipa: ipu3: Provide frame timestamps through IPU3EventUmang Jain
Pass in frame timestamps from IPU3 pipeline handler to IPU3 IPA via IPU3Event. Frame timestamps are helpful to IPA algorithms to convergence, by setting them via IPA stats. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-05-24ipa: ipu3: Introduce IPAConfigInfo in IPCUmang Jain
IPAConfigInfo is a consolidated data structure passed from IPU3 pipeline-handler to IPU3 IPA. The structure can be extended with additional parameters to accommodate the requirements of multiple IPU3 IPA modules. Adapt the in-tree IPU3 IPA to use IPAConfigInfo as well. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-04-22ipa: ipu3: Add support for IPU3 AEC/AGC algorithmJean-Michel Hautbois
Implement basic auto-exposure (AE) and auto-gain (AG) correction functions. The functions computeTargetExposure() and computeGain() are adapted from the Raspberry Pi AGC implementation to suit the IPU3 structures, and filtering is added to reduce visible stepsize when there are large exposure changes. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-04-22ipa: ipu3: Add support for IPU3 AWB algorithmJean-Michel Hautbois
The IPA will locally modify the parameters before they are passed down to the ImgU. Use a local parameter object to give a reference to those algorithms. Inherit from the Algorithm class to implement basic AWB functions. The configure() call will set exposure and gain to their minimum value, so while AGC is not there, the frames will be dark. Once AWB is done, a color temperature is estimated and a default CCM matrix will be used (yet to be tuned). Implement a basic "grey-world" AWB algorithm just for demonstration purpose. The BDS output size is passed by the pipeline handler to the IPA. The best grid is then calculated to maximize the number of pixels taken into account in each cells. As commented in the source code, it can be improved, as it has (at least) one limitation: if a cell is big (say 128 pixels wide) and indicated as saturated, it won't be taken into account at all. Maybe is it possible to have a smaller one, at the cost of a few pixels to lose, in which case we can center the grid using the x_start and y_start parameters. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-04-17ipa: ipu3: Move the IPA to the ipa::ipu3 namespaceKieran Bingham
Simplify name-spacing of the IPU3 components by placing it in the ipa::ipu3 namespace directly. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-03-24ipa: ipu3: Do not set controls during configureKieran Bingham
The configure operation is synchronous and should not send events back to the pipeline handler. If information needs to be returned from configure it should be handled through the interface directly. Move the initial call to setControls() out of configure() and into the start() method which is called after the IPA running_ state is updated. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-03-17libcamera: ipu3: Pass the BDS rectangle at IPA configure callJean-Michel Hautbois
The IPU3 IPA will need the BDS configuration when the AWB/AGC algorithm will be integrated. In order to do that, the configure() interface needs to be modified. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-02-19ipa: ipu3: Don't call SetControls for each frameJean-Michel Hautbois
Setting a default EXPOSURE and ANALOGUE_GAIN makes sense for the first frame but there is no need to force it for all frames. It will be called later by the 3A algorithms. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-02-16libcamera: IPAInterface: Replace C API with the new C++-only APIPaul Elder
Remove everything related to the C API, including ipa_context, ipa_context_wrapper, and IPAInterfaceWrapper. Also remove relevant documentation. ipaCreate() provided by IPA implementations, and createInterface() provided by IPAModule (wrapper around IPA implementation) both now return a C++ object IPAInterface instead of struct ipa_context. Although IPAInterfaceWrapper is the only component of libipa, the skeleton and build files for libipa are retained. After converting the C API to the C++-only API, make all pipeline handlers and IPAs use the new API. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- This is a combination of 21 commits: --- libcamera: IPAModule: Replace ipa_context with IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate_() and createInterface() return IPAInterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: ipa_context_wrapper: Remove ipa_context_wrapper Since ipa_context has been replaced with custom IPAInterfaces, it is not longer needed. Remove it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAInterface: remove ipa_context and functions from documentation Remove all the documentation related to ipa_context and the C IPA API, as well as the documentation about the functions in the IPAInterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> --- libcamera: IPAInterface: Remove all functions from IPAInterface Now that all the functions in the IPA interface are defined in the data definition file and a specialized IPAInterface is generated per pipeline handler, remove all the functions from the base IPAInterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAInterface: make ipaCreate return IPAInterface With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate return IPAinterface. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> --- ipa: remove IPAInterfaceWrapper As every pipeline has its own proxy, IPAInterfaceWrapper is no longer necessary. Remove it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Acked-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> --- libcamera: IPAProxy: Remove stop() override Since stop() is part of the IPA interface, and the IPA interface is now generated based on the data definition file per pipeline, this no longer needs to be overrided by the base IPAProxy. Remove it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAProxy, IPAManager: Switch to one-proxy-per-pipeline scheme IPAProxy is changed in two major ways: - Every pipeline has its own proxy, to support each pipeline's IPA interface - IPAProxy implementations always encapsulate IPA modules, and switch internally for isolation or threaded The IPAProxy registration mechanism is removed, as each pipeline will have its own proxy, so the pipeline can pass the specialized class name of the IPAProxy to the IPAManager for construction. IPAManager is changed accordingly to support these changes: - createIPA is a template function that takes an IPAProxy class, and always returns an IPAProxy - IPAManager no longer decides on isolation, and simply creates an IPAProxy instance while passing the isolation flag Consequently, the old IPAProxy classes (IPAProxyThread and IPAProxyLinux) are removed. The IPAInterfaceTest is updated to use the new IPAManager interface, and to construct a ProcessManager as no single global instance is created anymore. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> --- libcamera: IPAProxy: Add isolate parameter to create() Since IPAProxy implementations now always encapsulate IPA modules, add a parameter to create() to signal if the proxy should isolate the IPA or not. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline Now that each pipeline handler has its own IPAProxy implementation, make the IPAManager fetch the IPAProxy based on the pipeline handler name. Also, since the IPAProxy is used regardless of isolation or no isolation, remove the isolation check from the proxy selection. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> --- libcamera: IPAManager: add isolation flag to proxy creation When the IPA proxy is created, it needs to know whether to isolate or not. Feed the flag at creation of the IPA proxy. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAManager: Make createIPA return proxy directly Since every pipeline knows the type of the proxy that it needs, and since all IPAs are to be wrapped in a proxy, IPAManager no longer needs to search in the factory list to fetch the proxy factory to construct a factory. Instead, we define createIPA as a template function, and the pipeline handler can declare the proxy type when it calls createIPA. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: IPAProxy: Remove registration mechanism Implementations of IPA proxies use a registration mechanism to register themselves with the main IPA proxy factory. This registration declares static objects, causing a risk of things being constructed before the proper libcamera facilities are ready. Since each pipeline handler has its own IPA proxy and knows the type, it isn't necessary to have a proxy factory. Remove it to alleviate the risk of early construction. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread We have now changed the proxy from per-IPC mechanism to per-pipeline. The per-IPC mechanism proxies are thus no longer needed; remove them. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- tests: ipa_interface_test: Update to use new createIPA Update the IPA interface test to use the new createIPA function from IPAManager. Also create an instance of ProcessManager, as no single global instance is created automatically anymore. Update meson.build to to depend on the generated IPA interface headers. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: PipelineHandler: Remove IPA from base class Since pipeline handlers now have their own IPA interface types, it can no longer be defined in the base class, and each pipeline handler implementation must declare it and its type themselves. Remove it from the base class. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- ipa: raspberrypi: Add mojom data definition file Add a mojom data definition for raspberrypi pipeline handler's IPAs. This simplifies the API between the raspberrypi pipeline handler and the IPA, and is not a direct translation of what was used before with IPAOperationData. Also move the enums from raspberrypi.h to raspberrypi.mojom Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: pipeline, ipa: raspberrypi: Use new data definition Now that we can generate custom functions and data structures with mojo, switch the raspberrypi pipeline handler and IPA to use the custom data structures as defined in the mojom data definition file. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: pipeline, ipa: vimc: Support the new IPC mechanism Add support to vimc pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> --- libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism Add support to the rkisp1 pipeline handler and IPA for the new IPC mechanism. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> --- libcamera: pipeline, ipa: ipu3: Support the new IPC mechanism Add support to ipu3 pipeline handler and IPA for the new IPC mechanism. [Original version] Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> [Fixed commit message and small changes] Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-02-05libcamera: ipa: ipu3: Add an IPA skeleton for the IPU3 pipelineNiklas Söderlund
Add an empty IPA skeleton for the IPU3 pipeline. The skeleton IPA handles the flow of parameter and statistic buffers but does not read or write anything in the buffers. It also allows the IPA to set sensor controls but does not implement any logic to set optimal values and instead sets the V4L2 exposure and gain controls to max and keeps them at that setting. This IPA is meant as a base to allow the pipeline handler to be wired up to an IPA. The image algorithms can then later be added to the IPA independently from also having to add plumbing to the pipeline handler. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>