summaryrefslogtreecommitdiff
path: root/src/ipa/ipu3
AgeCommit message (Collapse)Author
2022-05-18ipa: libipa: Add frame context pointer in process()Umang Jain
Currently we have a single structure of IPAFrameContext but subsequently, we shall have a ring buffer (or similar) container to keep IPAFrameContext structures for each frame. It would be a hassle to query out the frame context required for process() (since they will reside in a ring buffer) by the IPA for each process. Hence, prepare the process() libipa template to accept a particular IPAFrameContext early on. As for this patch, we shall pass in the pointer as nullptr, so that the changes compile and keep working as-is. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-05-18ipa: ipu3: Rework IPAFrameContextUmang Jain
Currently, IPAFrameContext consolidates the values computed by the active state of the algorithms, along with the values applied on the sensor. Moving ahead, we want to have a frame context associated with each incoming request (or frame to be captured). This shouldn't necessarily be tied to "active state" of the algorithms hence: - Rename current IPAFrameContext -> IPAActiveState This will now reflect the latest active state of the algorithms and has nothing to do with any frame-related ops/values. - Re-instate IPAFrameContext with a sub-structure 'sensor' currently storing the exposure and gain value. Adapt the various access to the frame context to the new changes as described above. Subsequently, the re-instated IPAFrameContext will be extended to contain a frame number and ControlList to remember the incoming request controls provided by the application. A ring-buffer will be introduced to store these frame contexts for a certain number of frames. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-14ipa: ipu3: af: A not initialized frame ignore counter fixingKate Hsuan
A not initialized frame ignore counter (ignoreCounter_) makes the AF function not work since the ignore counter may start from a random negative number. The counter was set to kIgnoreFrame when AF is in prepare stage. Signed-off-by: Kate Hsuan <hpa@redhat.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: Simplify accumulations of y_itemsKieran Bingham
Simplify the accumulation of the total and variance with a ternary operator. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-04-09ipa: ipu3: af: Use Span for y_table_item_tKieran Bingham
Convert the y_table_item_t to a Span and use that for iteration when estimating variance of the table. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-04-09ipa: ipu3: af: Remove redundant memcpyKieran Bingham
The af statistics can be accessed directly from the mapped buffer. Remove the redundant memcpy, and simplify the call to afEstimateVariance(). Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: Use geometry classes to perform grid centeringKieran Bingham
Use our geometry classes for Rectangle, Size and Point to identify the region of interest for the autofocus, and center it on the BDS output. This will facilitate custom ROI being passed in through controls at a later time. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: Move constants to implementationKieran Bingham
A selection of constants are imported from ChromiumOS. Move these out of the header, and simplify their documentation. Further more, add a direct reference to the location they were obtained from. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kate Hsuan <hpa@redhat.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-04-09ipa: ipu3: af: enforce grid size restrictionsKieran Bingham
Provide enforcement of the selection of the block_{width,height}_log2 parameters to the capabilities of the hardware. While this selection is currently hardcoded to the minimum, providing the restriction now allows for further dynamic sizing in the future and documents the restrictions directly in code, making use of the already existing constants. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2022-04-08ipa: ipu3: Inline parseStatistics() into processStatsBuffer()Umang Jain
Since we have moved away from switch/case on the operation ID, there's little reason to split the operation in two functions. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-04-08ipa: ipu3: Inline fillParams() in fillParamsBuffer()Umang Jain
Since we have moved away from switch/case on the operation ID, there's little reason to split the operation in two functions. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-04-08ipa: ipu3: Replace event-based ops with dedicated functionsUmang Jain
The IPAIPU3 interface currently uses event-type based structures in order to communicate with the pipeline-handler (and vice-versa). Replace the event based structures with dedicated functions associated to each operation. The translated naming scheme of actions to signals are: ActionSetSensorControls => setSensorControls ActionParamFilled => paramsBufferReady ActionMetadataReady => metadataReady The translated naming scheme of events to dedicated functions are: EventProcessControls => queueRequest() EventStatReady => processStatsBuffer() EventFillParams => fillParamsBuffer() The dedicated functions are called from pipeline-handler to the IPA using IPC. These functions run asynchronously and when completed, the IPA emits the respective signals as stated above in the translated naming scheme. The EventProcessControls is translated to queueRequest() to bring symmetry to the IPU3 interface with other IPA interfaces. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-03-28ipa: ipu3: Ensure controls exists in before they are queriedUmang Jain
Add a validation check for sensor controls validateSensorControls() before they are queried in IPAIPU3::updateSessionConfiguration(). Fail the IPAIPU3::configure() if the required sensor controls are not found. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-03-28ipa: ipu3: Drop sensor controls private members from IPAIPU3Umang Jain
Drop exposure, gain private members from IPAIPU3 because the values are handled directly via IPAFrameContext. Move the default vblank value from IPAIPU3 to IPASessionConfiguration structure as it is a default static value not expected to change for a session. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-03-28ipa: ipu3: Consolidate querying of exposure and gain limitsUmang Jain
The exposure and gain limits are required for AGC configuration handled in IPAIPU3::updateSessionConfiguration(), which is happening already. Therefore the max/min private members in IPAIPU3 class for exposure/gain serve no use except setting initial values of exposure_ and gain_ members. Drop the max/min private members from IPAIPU3 class and set initial gain_ and exposure_ in IPAIPU3::updateSessionConfiguration(). Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-03-17ipa: ipu3: agc: Use existing local short-hand variable in configure()Laurent Pinchart
The configure() function has a local configuration variable referencing context.configuration for the purpose of shortening lines. Use it instead of context.configuration in the remaining locations, and constify it while at it as the configuration isn't meant to be modified. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-17ipa: ipu3: agc: Reset frame count when configuring AGCLaurent Pinchart
The frame count is used to skip the gain and exposure filtering when starting. It thus needs to be reset when configuring the algorithm, to avoid slower convergence when stopping and restarting. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-15ipa: ipu3: Send lens controls to pipeline handlerDaniel Scally
Pick up the focus value from the AF algorithm and send lens controls along in the frame context. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Daniel Scally <djrscally@gmail.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-15libcamera: ipa: Rename ctrls_ memberDaniel Scally
Now that we have added lens controls, rename the existiing member of the class to clarify that it relates to the sensor's controls. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Daniel Scally <djrscally@gmail.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-15libcamera: ipa: Add lens control member to ipu3 ipa interfaceDaniel Scally
Add further members to the ipu3 ipa interface that will hold lens controls passed in by configInfo Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Daniel Scally <djrscally@gmail.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-15ipa: ipu3: af: Auto focus for dw9719 Surface Go2 VCMKate Hsuan
Since VCM for surface Go 2 (dw9719) had been successfully driven, this Af module can be used to control the VCM and determine the focus value based on the IPU3 AF state. Based on the values from the IPU3 AF buffer, the variance of each focus step is determined and a greedy approach is used to find the maximum variance of the AF state and an appropriate focus value. The grid configuration is implemented as a context. Also, the grid parameter- AF_MIN_BLOCK_WIDTH is set to 4 (default is 3) since if the default value is used, x_start (x_start > 640) will be at an incorrect location of the image (rightmost of the sensor). Signed-off-by: Kate Hsuan <hpa@redhat.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-03-11ipa: ipu3: awb: Clamp gain valuesJean-Michel Hautbois
The gain values are coded as u3.13 fixed point values, ie they can not be more than 8. Clamp the values in order to avoid any off limits value which could make the IPU3 behave in a weird manner. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2022-03-11ipa: ipu3: agc: Introduce lineDuration in IPASessionConfigurationJean-Michel Hautbois
Instead of having a local cached value for line duration, store it in the IPASessionConfiguration::sensor structure. While at it, configure the default analogue gain and shutter speed to controlled fixed values. The latter is set to be 10ms as it will in most cases be close to the one needed, making the AGC faster to converge. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2022-03-11ipa: ipu3: Shorten exposure and gain linesJean-Michel Hautbois
When the effective sensor values are stored during the EventStatReady event, the lines are long. Fix it. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2022-03-11ipa: ipu3: Return filtered valueJean-Michel Hautbois
When the current exposure value is calculated, it is cached and used by filterExposure(). Use private filteredExposure_ and pass currentExposure as a parameter. In order to limit the use of filteredExposure_, return the value from filterExposure(). While at it, remove a stale comment. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2021-12-07libcamera: Use utils::abs_diff()Laurent Pinchart
Use the new utils::abs_diff() function where appropriate to replace manual implementations. While at it fix a header ordering issue in src/libcamera/pipeline/raspberrypi/raspberrypi.cpp. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-30ipa: ipu3: Rectify gain value reporting in request metadataUmang Jain
Report value of sensor's gain pertaining to the current request completion, as that is the gain value the request completed with. The Agc algorithm processes the gain value for incoming next request hence it should not be reported in request's metadata. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-29ipa: libipa: Introduce Algorithm class templateJean-Michel Hautbois
The algorithms are using the same function names with specialized parameters. Instead of duplicating code, introduce a libipa Algorithm class which implements a base class with template parameters in libipa, and use it in each IPA. As we now won't need an algorithm class for each IPA, move the documentation to libipa, and make it agnostic of the IPA used. While at it, fix the IPU3::Algorithm::Awb documentation. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-29ipa: Do not modify the sensor limitsJean-Michel Hautbois
The driver is responsible for setting the proper limits for its controls. The IMX219 has an analogue gain of 1.0 when the gain code is set to 0, therefore we can not clamp to a minimum gain code of 1. Rework this for both IPU3 and RkISP1, for both Exposure and Gain controls. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-24ipa: ipu3: Convert to pragma onceKieran Bingham
Remove the verbose #ifndef/#define/#endif pattern for maintaining header idempotency, and replace it with a simple #pragma once. This simplifies the headers, and prevents redundant changes when header files get moved. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-11-23ipa: ipu3: Fix IPAContext documentationJean-Michel Hautbois
The shutter speed limits and analogue gains limits are in the agc structure in IPASessionConfiguration. They are badly mentioned as being part of the grid structure, fix it. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Saturate the averages when computing relative luminanceLaurent Pinchart
The relative luminance is calculated using an iterative process to account for saturation in the sensor, as multiplying pixels by a gain doesn't increase the relative luminance by the same factor if some regions are saturated. Relative luminance estimation doesn't apply a saturation, which produces a value that doesn't match what the sensor will output, and defeats the point of the iterative process. Fix it. Fixes: f8f07f9468c6 ("ipa: ipu3: agc: Improve gain calculation") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Return the inter-quantile mean from measureBrightness()Laurent Pinchart
The inter-quantile mean is a value that is computed as part of the AGC run. It doesn't need to be stored in a member variable. Return it from measureBrightness(), which makes the flow of data easier to follow. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Rename currentYGainLaurent Pinchart
The "current" prefix in the currentYGain variable name is confusing: - In Agc::estimateLuminance(), the variable contains the gain to be applied to the image, which is neither a "current" gain nor a "Y" gain. Rename it to "gain". - In Agc::computeExposure(), the variable contains the gain computed by the relative luminance method, so rename it to "yGain". While at it, rename variables to match the libcamera coding style. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Standardize vocabulary on "relative luminance"Laurent Pinchart
The AGC computes the average relative luminance of the frame and calls the value "normalized luma", "brightness" or "initialY". The latter is the most accurate term, as the relative luminance is abbreviated Y, but the "initial" prefix isn't accurate. Standardize the vocabulary on "relative luminance" in code and comments, abbreviating it to Y when needed. While at it, rename variables to match the libcamera coding style. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-23ipa: ipu3: agc: Drop kMaxLuminance constantLaurent Pinchart
The kMaxLuminance constant is badly named, it's not a maximum luminance, but the maximum integer value output by the AWB statistics engine for per-channel averages. The constant is used in a single place, hardcoding the value is actually more readable. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-16ipa: ipu3: agc: Remove the threshold for the histogram calculationJean-Michel Hautbois
Until commit f8f07f9468c6 (ipa: ipu3: agc: Improve gain calculation) the gain to apply on the exposure value was only using the histogram. Now that the global brightness of the frame is estimated too, we don't need to remove part of the saturated pixels from the equation anymore. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-15ipa: ipu3: Set the AnalogueGain controlJean-Michel Hautbois
We can report the controls::AnalogueGain metadata now that AGC is updating it correctly. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Move ExposureTime to IPAJean-Michel Hautbois
Now that we have the exposure time calculated, report it in the controls::ExposureTime and don't use the pipeline handler for it anymore. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Cache line duration at configure callJean-Michel Hautbois
We use the line duration several times in the IPAIPU3. Instead of recalculating it each time, cache the value as a utils::Duration. ExposureTime and FrameDuration units are in micro-seconds, cast the Duration accordingly. As sensorInfo is no longer used in updateSessionConfiguration remove the reference to it. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Send color temperature in the metadataJean-Michel Hautbois
Now that the color temperature is updated per-frame, use the value and set the corresponding controls::ColourTemperature. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: awb: Add support for color temperatureJean-Michel Hautbois
The AWB estimates the color temperature, but it is not used at all. It can be useful for debug purpose at least, but also for lux estimation later, to be able to know the temperature estimated for a given frame. Add a new member to the IPAFrameContext::awb for this purpose, and update the value in AWB. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Store exposure in units of timeJean-Michel Hautbois
The minimum and maximum exposure are stored in lines. Replace it by values in time to simplify the calculations. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Update previous exposure valueJean-Michel Hautbois
Previously, the exposure value was calculated based on the estimated shutter time and gain applied. Now that we have the real values for the current frame, use those before estimating the next one and rename the variable accordingly. As the exposure value is updated in the beginning of the computation, there is no need to initialize effectiveExposureValue anymore in the configure call, and it can be a local variable and not a class variable anymore. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Improve gain calculationJean-Michel Hautbois
When an image is partially saturated, its brightness is not increasing linearly when the shutter time or gain increases. It is a big issue with a backlight as the algorithm is fading to darkness right now. Introduce a function to estimate the brightness of the frame, based on the current exposure/gain and loop on it several times to estimate it again and approach the non linear function. Inspired-by: 7de5506c30b3 ("libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images") Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Refactor ev gain calculation and testingJean-Michel Hautbois
When we compute the new gain, we use the iqMean_ and estimate an exposure value gain to apply. Return early when the gain is less than 1%. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Compute the gain for each frameJean-Michel Hautbois
Now that we have the real exposure applied at each frame, remove the early return based on a frame counter and compute the gain for each frame. Introduce a number of startup frames during which the filter speed is 1.0, meaning we apply instantly the exposure value calculated and not a slower filtered one. This is used to have a faster convergence, and those frames may be dropped in a future development to hide the convergance process from the viewer. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: agc: Limit the number of saturated cellsJean-Michel Hautbois
When the histogram is calculated, we check if a cell is saturated or not before cumulating its green value. This is wrong, and it can lead to an empty histogram in case of a fully saturated frame. Use a constant to limit the amount of pixels within a cell before considering it saturated. If at the end of the loop we still have an empty histogram, then make it a fully saturated one. Bug: https://bugs.libcamera.org/show_bug.cgi?id=84 Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Use sensor controls to update frameContextJean-Michel Hautbois
The pipeline handler populates the new sensorControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Make those values the frameContext::sensor values for the frame when the EventStatReady event is received. AGC also needs to use frameContext.sensor as its input values and frameContext.agc as its output values. Modify computeExposure by passing it the frameContext instead of individual exposure and gain values. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15ipa: ipu3: Extend ipu3 ipa interface for sensor controlsHan-Lin Chen
IPU3Event and IPU3Action use single ControlList for both libcamera and V4L2 controls, and it's content could be either one based on the context. Extend IPU3Event and IPU3Action for sensor V4L2 controls, and preserve the original one for only libcamera Controls to make the content of an event more specific. Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org> [Jean-Michel: remove lensControls from the original patch] Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>