summaryrefslogtreecommitdiff
path: root/src/ipa/ipu3/algorithms
AgeCommit message (Collapse)Author
2024-05-09libcamera: Drop remaining file name from header comment blocksLaurent Pinchart
Source files in libcamera start by a comment block header, which includes the file name and a one-line description of the file contents. While the latter is useful to get a quick overview of the file contents at a glance, the former is mostly a source of inconvenience. The name in the comments can easily get out of sync with the file name when files are renamed, and copy & paste during development have often lead to incorrect names being used to start with. Readers of the source code are expected to know which file they're looking it. Drop the file name from the header comment blocks in all remaining locations that were not caught by the automated script as they are out of sync with the file name. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
2024-05-08libcamera: Drop file name from header comment blocksLaurent Pinchart
Source files in libcamera start by a comment block header, which includes the file name and a one-line description of the file contents. While the latter is useful to get a quick overview of the file contents at a glance, the former is mostly a source of inconvenience. The name in the comments can easily get out of sync with the file name when files are renamed, and copy & paste during development have often lead to incorrect names being used to start with. Readers of the source code are expected to know which file they're looking it. Drop the file name from the header comment block. The change was generated with the following script: ---------------------------------------- dirs="include/libcamera src test utils" declare -rA patterns=( ['c']=' \* ' ['cpp']=' \* ' ['h']=' \* ' ['py']='# ' ['sh']='# ' ) for ext in ${!patterns[@]} ; do files=$(for dir in $dirs ; do find $dir -name "*.${ext}" ; done) pattern=${patterns[${ext}]} for file in $files ; do name=$(basename ${file}) sed -i "s/^\(${pattern}\)${name} - /\1/" "$file" done done ---------------------------------------- This misses several files that are out of sync with the comment block header. Those will be addressed separately and manually. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
2024-05-08ipa: ipu3: Remove bespoke AGC functions from IPU3Daniel Scally
Now that the IPU3's Agc is derived from MeanLuminanceAgc we can delete all the unecessary bespoke functions. Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-05-08ipa: ipu3: Derive ipu3::algorithms::Agc from AgcMeanLuminanceDaniel Scally
In preparation for switching to a derivation of AgcMeanLuminance, add a function to parse and store the statistics for easy retrieval in an overriding estimateLuminance() function. Now that we have a MeanLuminanceAgc class that centralises our AEGC algorithm, derive the IPU3's Agc class from it and plumb in the necessary framework to enable it to be used. For simplicity's sake this commit switches the algorithm to use the derived class, but does not remove the bespoke functions at this time. Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-06-06ipa: ipu3: agc: Drop hard-codec analogue gain maxJacopo Mondi
As the sensor's analogue gain range is known, drop the arbitrary maximum limit for the sensor analogue gain. Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com> Tested-by: Daniel Scally <dan.scally@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-09ipa: Sort algorithm operations based on calling orderLaurent Pinchart
Reorder functions in the base ipa::Algorithm and its derived classes to match the calling order: queueRequest(), prepare() and process(). This makes the code flow easier to read. No functional change intended. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-10-24ipa: ipu3: Fill AGC and AWB metadata in algorithmsLaurent Pinchart
Fill the frame metadata in the AGC and AWB algorithm's prepare() function. This removes the need to fill metadata manually in the IPA module's processStatsBuffer() function. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-10-24ipa: Pass metadata to Algorithm::process() functionLaurent Pinchart
Extend the Algorithm::process() function with a metadata control list, to be filled by individual algorithms with frame metadata. Update the rkisp1 and ipu3 IPA modules accordingly, and drop the dead code in the IPARkISP1::prepareMetadata() function while at it. This only creates the infrastructure, filling metadata in individual algorithms will be handled separately. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-10-19ipa: Drop period at end of \brief or \paramLaurent Pinchart
A few periods crept in at the end of Doxygen \brief or \param statements in the src/ipa/ directory. Remove them all in one go. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-09-28ipa: libipa: algorithm: process(): Pass frame numberKieran Bingham
Pass the frame number of the current frame being processed. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-09-28ipa: libipa: algorithm: prepare(): Pass frame and frame ContextKieran Bingham
Pass the current frame number, and the current FrameContext for calls to prepare. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-09-27ipa: libipa: Pass a reference instead of pointer to Algorithm::process()Laurent Pinchart
Frame contexts will become the core component of IPA modules, always available to functions of the algorithms. To indicate and prepare for this, turn the frame context pointer passed to Algorithm::process() into a reference. The RkISP1 IPA module doesn't use frame contexts yet, so pass a dummy context for now. While at it, drop an unneeded [[maybe_unused]] from Agc::process() and add a missing parameter documentation for the frameContext argument to Awb::process(). Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-09-27ipa: ipu3: af: Pass context reference to afIsOutOfFocus()Laurent Pinchart
Avoid copying the whole IPA context by passing a reference to the Af::afIsOutOfFocus() function. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-09-27ipa: ipu3: Fix style of Doxygen comment blocksLaurent Pinchart
Fix various issues in Doxygen comment blocks: - \param requires an [in] or [out] tag - \param must come before the body of the documetation - Drop leftover \param for argument that has been removed - Rename coarseSearchStep to kCoarseSearchStep - White space and line wrap Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-08-09ipa: ipu3: Register algorithmsLaurent Pinchart
To prepare for dynamic instantiation of algorithms from the tuning file, register the algorithms with the Module class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-07-20libcamera: Remove extra ':' after '\todo'Laurent Pinchart
The doxygen '\todo' directory doesn't need to be followed by a colon, yet a few strayed occurrences have made their way in. Fix them. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-06-29ipa: libipa: Introduce a Module class templateLaurent Pinchart
libipa defines an abstract Algorithm class template that is specialized by IPA modules. IPA modules then instantiate and manage algorithms internally, without help from libipa. With ongoing work on tuning data support for the RkISP1, and future similar work for the IPU3, more code duplication for algorithms management is expected. To address this and share code between multiple IPA modules, introduce a new Module class template that will define and manage top-level concepts for the IPA module. The Module class template needs to be specialized with the same types as the Algorithm class. To avoid manual specialization of both classes, store the types in the Module class, and replace the template arguments of the Algorithm class with a single Module argument from which the other types are retrieved. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-06-18ipa: ipu3: awb: Correct the gains calculationJean-Michel Hautbois
The factor used right now in the IPU3 is 8192, as a multiplier of the estimated gain. This is wrong, as the isp is adding 1.0 to the gain applied, ie Pout = { Pin * (1 + Gx) }. Fix it, and to ease the reading, introduce a small helper function. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-05-18ipa: ipu3: Put IPAFrameContext(s) in a ring bufferUmang Jain
Instead of having one frame context constantly being updated, this patch aims to introduce per-frame IPAFrameContext which are stored in a ring buffer. Whenever a request is queued, a new IPAFrameContext is created and inserted into the ring buffer. The IPAFrameContext structure itself has been slightly extended to store a frame id and a ControlList for incoming frame controls (sent in by the application). The next step would be to read and set these controls whenever the request is actually queued to the hardware. Since now we are working in multiples of IPAFrameContext, the Algorithm::process() will actually take in a IPAFrameContext pointer (as opposed to a nullptr while preparing for this change). Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-05-18ipa: libipa: Add frame context pointer in process()Umang Jain
Currently we have a single structure of IPAFrameContext but subsequently, we shall have a ring buffer (or similar) container to keep IPAFrameContext structures for each frame. It would be a hassle to query out the frame context required for process() (since they will reside in a ring buffer) by the IPA for each process. Hence, prepare the process() libipa template to accept a particular IPAFrameContext early on. As for this patch, we shall pass in the pointer as nullptr, so that the changes compile and keep working as-is. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-05-18ipa: ipu3: Rework IPAFrameContextUmang Jain
Currently, IPAFrameContext consolidates the values computed by the active state of the algorithms, along with the values applied on the sensor. Moving ahead, we want to have a frame context associated with each incoming request (or frame to be captured). This shouldn't necessarily be tied to "active state" of the algorithms hence: - Rename current IPAFrameContext -> IPAActiveState This will now reflect the latest active state of the algorithms and has nothing to do with any frame-related ops/values. - Re-instate IPAFrameContext with a sub-structure 'sensor' currently storing the exposure and gain value. Adapt the various access to the frame context to the new changes as described above. Subsequently, the re-instated IPAFrameContext will be extended to contain a frame number and ControlList to remember the incoming request controls provided by the application. A ring-buffer will be introduced to store these frame contexts for a certain number of frames. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-14ipa: ipu3: af: A not initialized frame ignore counter fixingKate Hsuan
A not initialized frame ignore counter (ignoreCounter_) makes the AF function not work since the ignore counter may start from a random negative number. The counter was set to kIgnoreFrame when AF is in prepare stage. Signed-off-by: Kate Hsuan <hpa@redhat.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: Simplify accumulations of y_itemsKieran Bingham
Simplify the accumulation of the total and variance with a ternary operator. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-04-09ipa: ipu3: af: Use Span for y_table_item_tKieran Bingham
Convert the y_table_item_t to a Span and use that for iteration when estimating variance of the table. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-04-09ipa: ipu3: af: Remove redundant memcpyKieran Bingham
The af statistics can be accessed directly from the mapped buffer. Remove the redundant memcpy, and simplify the call to afEstimateVariance(). Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: Use geometry classes to perform grid centeringKieran Bingham
Use our geometry classes for Rectangle, Size and Point to identify the region of interest for the autofocus, and center it on the BDS output. This will facilitate custom ROI being passed in through controls at a later time. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: Move constants to implementationKieran Bingham
A selection of constants are imported from ChromiumOS. Move these out of the header, and simplify their documentation. Further more, add a direct reference to the location they were obtained from. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: enforce grid size restrictionsKieran Bingham
Provide enforcement of the selection of the block_{width,height}_log2 parameters to the capabilities of the hardware. While this selection is currently hardcoded to the minimum, providing the restriction now allows for further dynamic sizing in the future and documents the restrictions directly in code, making use of the already existing constants. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2022-03-17ipa: ipu3: agc: Use existing local short-hand variable in configure()Laurent Pinchart
The configure() function has a local configuration variable referencing context.configuration for the purpose of shortening lines. Use it instead of context.configuration in the remaining locations, and constify it while at it as the configuration isn't meant to be modified. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-17ipa: ipu3: agc: Reset frame count when configuring AGCLaurent Pinchart
The frame count is used to skip the gain and exposure filtering when starting. It thus needs to be reset when configuring the algorithm, to avoid slower convergence when stopping and restarting. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-15ipa: ipu3: af: Auto focus for dw9719 Surface Go2 VCMKate Hsuan
Since VCM for surface Go 2 (dw9719) had been successfully driven, this Af module can be used to control the VCM and determine the focus value based on the IPU3 AF state. Based on the values from the IPU3 AF buffer, the variance of each focus step is determined and a greedy approach is used to find the maximum variance of the AF state and an appropriate focus value. The grid configuration is implemented as a context. Also, the grid parameter- AF_MIN_BLOCK_WIDTH is set to 4 (default is 3) since if the default value is used, x_start (x_start > 640) will be at an incorrect location of the image (rightmost of the sensor). Signed-off-by: Kate Hsuan <hpa@redhat.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-11ipa: ipu3: awb: Clamp gain valuesJean-Michel Hautbois
The gain values are coded as u3.13 fixed point values, ie they can not be more than 8. Clamp the values in order to avoid any off limits value which could make the IPU3 behave in a weird manner. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2022-03-11ipa: ipu3: agc: Introduce lineDuration in IPASessionConfigurationJean-Michel Hautbois
Instead of having a local cached value for line duration, store it in the IPASessionConfiguration::sensor structure. While at it, configure the default analogue gain and shutter speed to controlled fixed values. The latter is set to be 10ms as it will in most cases be close to the one needed, making the AGC faster to converge. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2022-03-11ipa: ipu3: Return filtered valueJean-Michel Hautbois
When the current exposure value is calculated, it is cached and used by filterExposure(). Use private filteredExposure_ and pass currentExposure as a parameter. In order to limit the use of filteredExposure_, return the value from filterExposure(). While at it, remove a stale comment. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2021-12-07libcamera: Use utils::abs_diff()Laurent Pinchart
Use the new utils::abs_diff() function where appropriate to replace manual implementations. While at it fix a header ordering issue in src/libcamera/pipeline/raspberrypi/raspberrypi.cpp. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-29ipa: libipa: Introduce Algorithm class templateJean-Michel Hautbois
The algorithms are using the same function names with specialized parameters. Instead of duplicating code, introduce a libipa Algorithm class which implements a base class with template parameters in libipa, and use it in each IPA. As we now won't need an algorithm class for each IPA, move the documentation to libipa, and make it agnostic of the IPA used. While at it, fix the IPU3::Algorithm::Awb documentation. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-24ipa: ipu3: Convert to pragma onceKieran Bingham
Remove the verbose #ifndef/#define/#endif pattern for maintaining header idempotency, and replace it with a simple #pragma once. This simplifies the headers, and prevents redundant changes when header files get moved. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Saturate the averages when computing relative luminanceLaurent Pinchart
The relative luminance is calculated using an iterative process to account for saturation in the sensor, as multiplying pixels by a gain doesn't increase the relative luminance by the same factor if some regions are saturated. Relative luminance estimation doesn't apply a saturation, which produces a value that doesn't match what the sensor will output, and defeats the point of the iterative process. Fix it. Fixes: f8f07f9468c6 ("ipa: ipu3: agc: Improve gain calculation") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Return the inter-quantile mean from measureBrightness()Laurent Pinchart
The inter-quantile mean is a value that is computed as part of the AGC run. It doesn't need to be stored in a member variable. Return it from measureBrightness(), which makes the flow of data easier to follow. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Rename currentYGainLaurent Pinchart
The "current" prefix in the currentYGain variable name is confusing: - In Agc::estimateLuminance(), the variable contains the gain to be applied to the image, which is neither a "current" gain nor a "Y" gain. Rename it to "gain". - In Agc::computeExposure(), the variable contains the gain computed by the relative luminance method, so rename it to "yGain". While at it, rename variables to match the libcamera coding style. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Standardize vocabulary on "relative luminance"Laurent Pinchart
The AGC computes the average relative luminance of the frame and calls the value "normalized luma", "brightness" or "initialY". The latter is the most accurate term, as the relative luminance is abbreviated Y, but the "initial" prefix isn't accurate. Standardize the vocabulary on "relative luminance" in code and comments, abbreviating it to Y when needed. While at it, rename variables to match the libcamera coding style. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Drop kMaxLuminance constantLaurent Pinchart
The kMaxLuminance constant is badly named, it's not a maximum luminance, but the maximum integer value output by the AWB statistics engine for per-channel averages. The constant is used in a single place, hardcoding the value is actually more readable. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-16ipa: ipu3: agc: Remove the threshold for the histogram calculationJean-Michel Hautbois
Until commit f8f07f9468c6 (ipa: ipu3: agc: Improve gain calculation) the gain to apply on the exposure value was only using the histogram. Now that the global brightness of the frame is estimated too, we don't need to remove part of the saturated pixels from the equation anymore. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-15ipa: ipu3: awb: Add support for color temperatureJean-Michel Hautbois
The AWB estimates the color temperature, but it is not used at all. It can be useful for debug purpose at least, but also for lux estimation later, to be able to know the temperature estimated for a given frame. Add a new member to the IPAFrameContext::awb for this purpose, and update the value in AWB. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Store exposure in units of timeJean-Michel Hautbois
The minimum and maximum exposure are stored in lines. Replace it by values in time to simplify the calculations. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Update previous exposure valueJean-Michel Hautbois
Previously, the exposure value was calculated based on the estimated shutter time and gain applied. Now that we have the real values for the current frame, use those before estimating the next one and rename the variable accordingly. As the exposure value is updated in the beginning of the computation, there is no need to initialize effectiveExposureValue anymore in the configure call, and it can be a local variable and not a class variable anymore. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Improve gain calculationJean-Michel Hautbois
When an image is partially saturated, its brightness is not increasing linearly when the shutter time or gain increases. It is a big issue with a backlight as the algorithm is fading to darkness right now. Introduce a function to estimate the brightness of the frame, based on the current exposure/gain and loop on it several times to estimate it again and approach the non linear function. Inspired-by: 7de5506c30b3 ("libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images") Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Refactor ev gain calculation and testingJean-Michel Hautbois
When we compute the new gain, we use the iqMean_ and estimate an exposure value gain to apply. Return early when the gain is less than 1%. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Compute the gain for each frameJean-Michel Hautbois
Now that we have the real exposure applied at each frame, remove the early return based on a frame counter and compute the gain for each frame. Introduce a number of startup frames during which the filter speed is 1.0, meaning we apply instantly the exposure value calculated and not a slower filtered one. This is used to have a faster convergence, and those frames may be dropped in a future development to hide the convergance process from the viewer. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Limit the number of saturated cellsJean-Michel Hautbois
When the histogram is calculated, we check if a cell is saturated or not before cumulating its green value. This is wrong, and it can lead to an empty histogram in case of a fully saturated frame. Use a constant to limit the amount of pixels within a cell before considering it saturated. If at the end of the loop we still have an empty histogram, then make it a fully saturated one. Bug: https://bugs.libcamera.org/show_bug.cgi?id=84 Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>