summaryrefslogtreecommitdiff
path: root/src/ipa/ipu3/algorithms
AgeCommit message (Collapse)Author
2021-10-26ipa: ipu3: tonemapping: Generate the LUT only on gamma changeJean-Michel Hautbois
The tone mapping algorithm calculates the gamma curve for every frame, regardless of whether the gamma value has changed or not. This issue is exasperated as we currently hardcode the gamma to a single value. Optimise the implementation to only recalculate the look up table when the gamma setting is changed, and store the gamma setting of the LUT curve as part of the IPA context. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26ipa: ipu3: agc: Document AGC mean-based algorithmJean-Michel Hautbois
The AGC class was not documented while developing. Extend that to reference the origins of the implementation, and improve the descriptions on how the algorithm operates internally. While at it, rename the functions which have bad names. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26ipa: ipu3: awb: Reword accumulator documentationJean-Michel Hautbois
Now that we moved the diagram into the AWB class documentation, reword the accumulator documentation to make it clear it is not meant to be used only in AWB. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26ipa: ipu3: awb: Add AWB class documentationJean-Michel Hautbois
The AWB algorithm is based on the Grey world algorithm and uses the statistics generated by the ImgU for that. Explain how it uses those, and reference the original algorithm at the same time. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: Use sensor limits for analogue gainJean-Michel Hautbois
Instead of using constants for the analogue gains limits, use the minimum and maximum from the configured sensor. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Remove unused variablesJean-Michel Hautbois
We currently control the exposure value by the shutter speed and the analogue gain. We can't use the digital gain to have more than the maximum exposure value calculated because we are not controlling it. Remove unused code associated with this digital gain. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Refactor condition on exposure correctionJean-Michel Hautbois
Simplify the reading by removing one level of indentation to return early when the change is small between two calls. Reword the LOG() message when we are correctly exposed, and move the lastFrame_ variable to update it even if the change is small. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Introduce previous exposure valueJean-Michel Hautbois
We need to calculate the gain on the previous exposure value calculated. Now that we initialise the exposure and gain values in configure(), we know the initial exposure value, and we can set it before any loop is running. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Rename gains properlyJean-Michel Hautbois
We have mixed terms between gain, analogue gain and the exposure value gain. Make it clear when we are using the analogue gain from the sensor, and when we are using the calculated gain to be applied to the exposure value to reach the target. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Simplify division of exposure/gainJean-Michel Hautbois
Until now, the algorithm makes complex assumptions when dividing the exposure and analogue gain values. Instead, use a simpler clamping of the shutter speed first, and then of the analogue gain, based on the limits configured. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Use filtered exposure valuesJean-Michel Hautbois
We are filtering the exposure value to limit the gain to apply, but we are not using the result. Fix it. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Change analogue gain limitsJean-Michel Hautbois
The gains are currently set as a uint32_t while the analogue gain is passed as a double. We also have a default maximum analogue gain of 15 which is quite high for a number of sensors. Use a maximum value of 8 which should really be configured by the IPA and not fixed as it is now. While at it make it a double. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Change exposure limitsJean-Michel Hautbois
We are using arbitrary constants for the exposure limit in a number of lines. Instead of using static constants for those, use the limits of the sensor passed in IPASessionConfiguration and cache those. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: agc: Rename exposure values properlyJean-Michel Hautbois
The exposure value is filtered in filterExposure() using the currentExposure_ and setting a prevExposure_ variable. This is misnamed as it is not the previous exposure, but a filtered value. Rename it accordingly. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: awb: Change minimal green threshold valueJean-Michel Hautbois
When zones are used for the grey world algorithm, they are only considered if their average green value is at least 32/255 to exclude zones that are too dark and don't provide relevant colour information (on the opposite side of the spectrum, saturated regions are excluded by the ImgU statistics engine). The algorithm requires a minimal number of zones that meet this criteria in order to run. Now that we correct the black level, the 32/255 minimal value is a bit high and prevents the algorithm for running in low-light conditions. Lower the value to 16/255 to fix it. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: awb: Use saturation under 90%Jean-Michel Hautbois
The AWB grey world algorithm tries to find a grey value and it can't do it on over-exposed images. To exclude those, the saturation ratio is used for each cell, and the cell is included only if this ratio is 0. Now that we have changed the threshold, more cells may be considered as partially saturated and excluded, preventing the algorithm from running efficiently. Change that behaviour, and consider 90% as a good enough ratio. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22ipa: ipu3: set frameContext before controlsJean-Michel Hautbois
The AGC frame context needs to be initialised correctly for the first iteration. Until now, the IPA uses the minimum exposure and gain values and caches those in local variables. In order to give the sensor limits to AGC, create a new structure in IPASessionConfiguration. Store the exposure in time (and not line duration) and the analogue gain after CameraSensorHelper conversion. Set the gain and exposure appropriately to the current values known to the IPA and remove the setting of exposure and gain in IPAIPU3 as those are now fully controlled by IPU3Agc. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22ipa: ipu3: awb: Set a threshold for the green saturationJean-Michel Hautbois
We can have a saturation ratio per cell, giving the percentage of pixels over a threshold within a cell where 100% is set to 0xff. The parameter structure 'ipu3_uapi_awb_config_s' contains four fields to set the threshold, one per channel. The blue field is also used to configure the ImgU and make it calculate the saturation ratio or not. Set a green value saturated when it is more than 230 (90% of the maximum value 255, coded as 8191). As this is the only channel used for AGC, there is no need to apply it to the other ones. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-19ipa: ipu3: agc: Remove "using" directive from agc.h headerLaurent Pinchart
"using" directives are harmful in headers, as they propagate the namespace short-circuit to all files that include the header, directly or indirectly. Drop the directive from agc.h, and use utils::Duration explicitly. While at it, shorten the namespace qualifier from libcamera::utils:: to utils:: in agc.cpp for Duration. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-10-19ipa: ipu3: awb: Don't pass member variable to member functionLaurent Pinchart
The Awb::generateZones() member function fills the zones vector passed as an argument, which is actually a member variable. Use it directly in the function. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-10-06ipa: ipu3: Replace ipa::ipu3::algorithms::Ipu3AwbCellJean-Michel Hautbois
The intel-ipu3.h public interface from the kernel does not define how to parse the statistics for a cell. This had to be identified by a process of reverse engineering, and later identifying the structures from [0] leading to our custom definition of struct Ipu3AwbCell. [0] https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h To improve the kernel interface, a proposal has been made to the linux-kernel [1] to incorporate the memory layout for each cell into the intel-ipu3 header directly. [1] https://lore.kernel.org/linux-media/20211005202019.253353-1-jeanmichel.hautbois@ideasonboard.com/ Update our local copy of the intel-ipu3.h to match the proposal and change the AGC and AWB algorithms to reference that structure directly, allowing us to remove the deprecated custom Ipu3AwbCell definition. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: agc: Rewrite and simplify the brightness loopJean-Michel Hautbois
Now that we know how the AWB statistics are formatted, use a simplified loop in processBrightness() to parse the green values and get the histogram. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Introduce Black Level CorrectionJean-Michel Hautbois
The pixels output by the camera normally include a black level, because sensors do not always report a signal level of '0' for black. Pixels at or below this level should be considered black and to achieve that, we need to substract an offset to all the pixels. This can be taken into account by reading the lowest value of a special region on sensors which is not exposed to light. This provides a substracting factor to be able to adjust the expected black levels in the resulting images. For a camera outputting 10-bit pixel values (in the range 0 to 1023) a typical black level might be 64. It is a fixed value, obtained by capturing a raw frame with minimum exposure and gain fixed to 1.0 while covering the sensor (the darker the better). We consider it good enough as a very first approximation, until we measure it during a tuning process and include it in a configuration file Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Use the line stride for the statsJean-Michel Hautbois
The statistics buffer 'ipu3_uapi_awb_raw_buffer' stores the ImgU calculation results in a buffer aligned horizontally to a multiple of 4 cells. The AWB loop should take care of it to add the proper offset between lines and avoid any staircase effect. It is no longer required to pass the grid configuration context to the private functions called from process() which simplifies the code flow. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Correct the gain multipliersJean-Michel Hautbois
The gains have a precision u3.13, range [0, 8[ which means that a gain multiplier value of 1.0 is represented as a multiplication by 8192 in the ImgU. Correct the gains as this was misunderstood in the first place. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Correct the relevant zones proportionJean-Michel Hautbois
The algorithm uses the statistics of a cell only if there is not too much saturated pixels in it. The grey world algorithm works fine when there are a limited number of outliers. Consider a valid zone to be at least 80% of unsaturated cells in it. This value could very well be configurable, and make the algorithm more or less tolerant. While at it, implement it in a configure() call as it will not change during execution, and cache the cellsPerZone values estimated with std::round as we are using cmath. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: awb: Make the naming consistentJean-Michel Hautbois
The variables mix the terms cell, region and zone. It can confuse the reader, and make the algorithm more difficult to follow. Rename the local variables to be consistent with their definitions: - Cells are defined in Pixels - Zones are defined in Cells There is no "region" as such, so replace it with the correct term. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Change Accumulator structure layoutJean-Michel Hautbois
The pixel component sums for the Accumulator are inconsistent with other similar structures such as the IPAFrameContext::awb::gains. Group the red, green, and blue sums together in a struct and store them as uint64_t to reduce potential architectural differences. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Rename IspStatsRegion to AccumulatorJean-Michel Hautbois
The IspStatsRegion structure was introduced as an attempt to prepare for a generic AWB algorithm structure. The structure name by itself is not explicit and it is too optimistic to try and make a generic one for now. Its role is to accumulate the pixels in a given zone. Rename it to accumulator, and remove the uncounted field at the same time. It is always possible to know how many pixels are not relevant for the algorithm by calculating total-counted. The uncounted field was only declared and not used. Amend the documentation accordingly. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06ipa: ipu3: Move the AWB stats structuresJean-Michel Hautbois
The structure Ipu3AwbCell describes the AWB stats layout on the kernel side. We will need it to be used by the AGC algorithm to be introduced later, so let's make it visible from ipa::ipu3::algorithms and not only for the AWB class. The IspStatsRegion will be needed by AGC too, so let's move it in the same namespace too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-23ipa: ipu3: awb: remove local storage of the gridJean-Michel Hautbois
The IPASessionConfiguration now has the grid configuration stored. Use it it at prepare() and process() calls in AWB and pass it as a reference to the private functions when needed. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-23ipa: ipu3: agc: remove local storage of the gridJean-Michel Hautbois
The IPASessionConfiguration now has the grid configuration stored. Use it at process() call in AGC and pass it as a reference to the private functions when needed. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-20ipa: ipu3: Move IPU3 agc into algorithmsJean-Michel Hautbois
Now that the interface is properly used by the AGC class, move it into ipa::ipu3::algorithms and let the loops do the calls. As we need to exchange the exposure_ and gain_ by passing them through the FrameContext, use the calculated values in setControls() function to ease the reading. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Move IPU3 awb into algorithmsJean-Michel Hautbois
Now that the interface is properly used by the AWB class, move it into ipa::ipu3::algorithms and let the loops do the calls. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Introduce a modular tone mapping algorithmJean-Michel Hautbois
Introduce a new algorithm to manage the tone mapping handling of the IPU3. The initial algorithm is chosen to configure the gamma contrast curve which moves the implementation out of AWB for simplicity. As it is initialised with a default gamma value of 1.1, there is no need to use the default table at initialisation anymore. This demonstrates the way to use process() call when the EventStatReady comes in. The function calculates the LUT in the context of a frame, and when prepare() is called, the parameters are filled with the updated values. AGC is modified to take the new process interface into account. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: ipu3: Add the functions to the Algorithm classJean-Michel Hautbois
Introduce three functions in the Algorithm class to manage algorithms: - configure which is called when IPA is configured only - prepare called on EventFillParams event at each frame when the request is queued - process called on EventStatReady event at each frame completion when the statistics have been generated. The existing AGC implementation already has a function named process(), though it has different arguments. Adding the new virtual process() interface causes a compiler warning due to the AGC implementation overloading a virtual function, even though the overload can be resolved correctly. Temporarily disable the warning in this commit to maintain bisection until the AGC is converted to the new interface. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20ipa: move libipa::Algorithm to ipa/ipu3/algorithmsJean-Michel Hautbois
The abstract Algorithm class was originally placed in libipa as an attempt define a generic algorithm container. This was a little optimistic and pushed a bit far too early. Move the Algorithm class into the IPU3 which is the only user of the class, as we adapt it to support modular algorithm components for the IPU3. Not documenting the namespace may cause issues with Doxygen in libipa. The file libipa.cpp is thus created as an empty file for now, but we can leverage it in the future to add more global libipa documentation, and possibly code too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>