summaryrefslogtreecommitdiff
path: root/src/ipa
AgeCommit message (Collapse)Author
2023-03-07ipa: raspberrypi: Better heuristics for calculating Unicam timeoutNaushir Patuck
The existing mechanism of setting a timeout value simply uses the maximum possible frame length advertised by the sensor mode. This can be problematic when, for example, the IMX477 sensor can use a frame length of over 600 seconds. However, for typical usage the frame length will never go over several 100s of milliseconds, making the timeout very impractical. Store a list of the last 10 frame length values requested by the AGC. On startup, and at the end of every frame, take the maximum frame length value from this list and return that to the pipeline handler through the setCameraTimeoutValue() signal. This allows the timeout value to better track the actual sensor usage. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-03-07pipeline: ipa: raspberrypi: Change Unicam timeout handlingNaushir Patuck
Add an explicit helper function setCameraTimeout() in the pipeline handler to set the Unicam timeout value. This function is signalled from the IPA to set up an appropriate timeout. This replaces the maxSensorFrameLengthMs value parameter returned back from IPARPi::start(). Adjust the timeout to be 5x the maximum frame duration reported by the IPA. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-03-06ipa: rkisp1: lsc: Fix integer division errorJacopo Mondi
The RkISP1 implementation of the LensShadingCorrection algorithm has been made adaptive to the scene color temperature in commit 14c869c00fdd ("ipa: rkisp1: Take into account color temperature during LSC algorithm"). The LSC algorithm interpolates the correction factors using the table's reference color temperatures. When calculating the interpolation coefficients, an unintended integer division makes both coefficient zeros resulting in a completely black image. Fix this by type casting to double one of the division operands. Fixes: 14c869c00fdd ("ipa: rkisp1: Take into account color temperature during LSC algorithm") Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2023-02-09ipa: raspberrypi: Normalise region sums to 16-bitsNaushir Patuck
The VC4 ISP uses a pipeline bit-depth of 13-bits. The AGC algorithm needs to know this bit-depth when computing the Y value for the image. Instead of hardcoding the VC4 bit-depth in the AGC source code, normalise all region sums to 16-bits when filling the Statistics structure. AWB and ALSC are agnostic about pipeline depth, so do not need changing. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-02-09ipa: raspberrypi: Use the generic statistics structure in the algorithmsNaushir Patuck
Repurpose the StatisticsPtr type from being a shared_ptr<bcm2835_isp_stats> to shared_ptr<RPiController::Statistics>. This removes any hardware specific header files and structures from the algorithms source code. Add a new function in the Raspberry Pi IPA to populate the generic statistics structure from the values provided by the hardware in the bcm2835_isp_stats structure. Update the Lux, AWB, AGC, ALSC, Contrast, and Focus algorithms to use the generic statistics structure appropriately in their calculations. Additionally, remove references to any hardware specific headers and defines in these source files. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-02-09ipa: raspberrypi: awb: Delay release of the statistics bufferNaushir Patuck
Release the statistics buffer after running the through the AWB calculations. Only the "counted" statistics are copied out to a local structure, so keeping the statistics buffer allows the algorithm to see the "uncounted" statistics as well. This is currently handled by hard-coding the total number of statistics regions regions based on the structure definition in the bcm2835_isp_stats structure. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-02-09ipa: raspberrypi: histogram: Add a constructor for an empty histogramNaushir Patuck
Add a default constructor to the RPiController::Histogram class that creates an empty histogram. Since this is a cumulative histogram, push a value of 0 into the first (and only) bin to signify this. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-02-09ipa: raspberrypi: Generalise statisticsNaushir Patuck
At present, the controller algorithms access the bcm2835_isp_stats structure, which is hardware specific. It would be desirable to abstract out the statistics structure to remove hardware specific headers from the algorithms source files. Define a new templated RegionStats class that encompasses region based statistics generated by the ISP. For the VC4 ISP, this can be used to hold RGB sums and focus FoM values. Define a new Statistics structure that holds all the VC4 ISP statistics output. This includes AGC histograms, AGC/AWB region sums and focus FoM regions. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Add support for the Sony IMX708 sensorNick Hollinghurst
Add support for Raspberry Pi Camera 3 modules (Sony IMX708 camera sensor) to the Raspberry Pi IPA. These modules are available in either normal or wide angle lens, both with IR or no IR cut options, giving a total for 4 variants. Provide IQ tuning files for all four variants. The IMX708 camera helper additionally parses PDAF and HDR histogram data that is provided in the embedded data stream from Unicam. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Add lens position to DeviceStatusNaushir Patuck
Add the current frame's lens position (in dioptres) to the DeviceStatus structure. This value is obtained from the AfAlgorithm::getLensPosition() member function. Return this lens position back to the pipeline handler to store in the metadata field of the request. As a drive-by, fixup some inaccurate comments in the DeviceStatus structure. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Nick Hollinghurst nick.hollinghurst@raspberrypi.com Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: First version of autofocus algorithm using PDAFNick Hollinghurst
Provide the first version of the Raspberry Pi autofocus algorithm. This implementation uses a hybrid of contrast detect autofocus (CDAF) and phase detect autofocus (PDAF) statistics. PDAF is always preferred over CDAF due to having less "hunting" behavior. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Handle autofocus algorithm resultsNick Hollinghurst
Handle the results returned from the autofocus algorithm by updating lens actuator position to the desired value. Update the Request metadata with the algorithm status to provide back to the application. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Handle autofocus controlsNick Hollinghurst
Add handlers for all autofocus related libcamera controls. Translate these controls to the RPiController autofocus algorithm API. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Include autofocus controls in the IPA ControlInfoMapNaushir Patuck
Add all the autofocus controls handled by the IPA into the ControlInfoMap if a controllable lens actuator is present. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Reorder header file inclusionNaushir Patuck
Reorder the libcamera/request header file inclusion to match the libcamera guidelines. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30ipa: raspberrypi: Add autofocus algorithm interface headersNick Hollinghurst
Add a new AfAlgorithm class that defines the interface for the Raspberry Pi autofocus algorithm. Add an AfStatus structure that returns the results of the autofocus algorithm back to the IPA. Add a structure for PDAF statistics provided by supported sensors for use with an autofocus algorithm. This structure is currently based on the IMX708's PDAF "Type 1" statistics. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30pipeline: ipa: raspberrypi: Validate lens controlsNaushir Patuck
Pass the available lens controls to the IPA through the configure() function. Validate that the V4L2_CID_FOCUS_ABSOLUTE does exist. If it doesn't, log a warning message, and do not advertise focus related controls from the IPA. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30pipeline: ipa: raspberrypi: Replace entityControlsNaushir Patuck
Replace the legacy entityControls map passed into ipa::configure() with explicit fields for sensor and ISP controls. This removes any ambiguity over which set of controls corresponds with specific integer keys. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30pipeline: ipa: raspberrypi: Remove unused streamConfigNaushir Patuck
Remove the streamConfig parameter from the ipa::configure() call, it is never used. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-30pipeline: ipa: raspberrypi: Check if lens actuator is availableNaushir Patuck
Check if a lens actuator is available by the presense of the lens driver entity in the pipeline handler. Pass this result to the IPA on init. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-26libcamera: Add support for OmniVision OV8858Nicholas Roth
Support for the OmniVision OV8858 sensor is scheduled for inclusion in the Linux kernel in version v6.3. Add support for the sensor in libcamera by providing static properties and a camera sensor helper in libipa. The camera sensor helper expresses analogue gain increments in 1/128 step which differs from what is reported in the sensor documentation in section "5.8 manual exposure compensation/ manual gain compensation" [0] A more detailed analysis of the sensor gain model is reported at: https://patchwork.linuxtv.org/project/linux-media/patch/20221106171129.166892-2-nicholas@rothemail.net/#142267 Record with a \todo note a reference to discussion on the gain model implementation. Signed-off-by: Nicholas Roth <nicholas@rothemail.net> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-25ipa: rkisp1: Raise maximum analogue gainMikhail Rudenko
Omnivision OV4689 sensor driver exposes maximum analogue gain of 16x. Raise kMaxAnalogueGain to 16.0, so that the full gain range can be used. Signed-off-by: Mikhail Rudenko <mike.rudenko@gmail.com> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
2023-01-25ipa: rkisp1: Add OV4689 tuning fileMikhail Rudenko
Add a minimal tuning file for Omnivision OV4689, specifying black level subtraction level. Signed-off-by: Mikhail Rudenko <mike.rudenko@gmail.com> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
2023-01-25ipa: camera_sensor_helper: Add OV4689 supportMikhail Rudenko
Add support for Omnivision OV4689 image sensor to libipa. The sensor implements a linear gain equation with analogue gain control values 0-2047 corresponding to 0x-16x physical gain range. Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Signed-off-by: Mikhail Rudenko <mike.rudenko@gmail.com> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
2022-12-24libcamera: Simple typo fixesPavel Machek
Fix various typos in the code base. Signed-off-by: Pavel Machek <pavel@ucw.cz> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-12-22ipa: raspberrypi: Fix permissions on imx477_scientific.jsonLaurent Pinchart
The imx477_scientific.json file was added to the source tree with incorrect executable permission. Fix it. Fixes: bf66a1d25162 ("ipa: raspberrypi: Add a "scientific" tuning for the IMX477") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-12-22ipa: camera_sensor_helper: Add AR0521 supportJacopo Mondi
Add support for OnSemi AR0521 5Mpx image sensor to libipa. The sensor analogue gain is implemented as a coarse and a fine factor, with the coarse gain being a power of two and the fine gain being a value in the [1.0, 2.0[ range. The mapping between gain codes and gain values is tabulated in the datasheet, and the table values are very close but not identical to the mathematical model. Compute the gain using the model to keep the code shorter, if this causes precision issues the calculation could be replaced with a table. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-12-19ipa: raspberrypi: Add a "scientific" tuning for the IMX477Naushir Patuck
Add a tuning file for the IMX477 more suited to scientific applications. The key differences from the original tuning file are: - Disable ALSC block completely - Pure rec709 gamma curve, and no contrast enhance - New CT curve and CCMs based on the illumination spectrum of a black body radiator up to about 3600 K and the CIE illuminant D for higher color temperatures. Further details on the changes can be found at: https://forums.raspberrypi.com/viewtopic.php?t=343449 All credit for these changes go to Dr. Rolf Henkel. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Signed-off-by: Rolf Henkel <cpixip@pixelcircus.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-12-09ipa: rkisp1: Add additional default values to controlsRobert Mader
To make things easier for consumers. Related: https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests/1450 Signed-off-by: Robert Mader <robert.mader@collabora.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-11-30ipa: raspberrypi: imx477: Update tuning file for the latest camera modulesDavid Plowman
The latest camera modules have a very slightly different IR filter, so the tuning file is slightly revised to give best results with both old and new camera modules. The original tuning file is retained as imx477_v1.json in case anyone should wish to continue using it. Signed-off-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-11-30ipa: raspberrypi: Fix bug in IPA frame drop logicNaushir Patuck
Fix a bug in the IPA frame dropping (for rate control) logic, where the metadata for the current context was copied from itself (i.e. a no-op), instead of being copied from the previous context. This bug does not occur in normal conditions, only when running with a low exposure time and unconstrained framerate, which happens in a particular picamera2 test. Fixes: 546154b13433 ("pipeline: ipa: raspberrypi: Use IPA cookies") Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2022-11-29ipa: raspberrypi: agc: Fix digital gain calculation for manual modeNaushir Patuck
The digital gain calculation uses a total exposure value computed with the current AGC state. However, this is wrong in the case of manual shutter/gain controls, as the total exposure value used must be the value computed when the AGC sent the manual shutter/gain controls to the pipeline handler to action. To fix this, the IPA now adds the historical AgcStatus structure to the metadata (tagged with "agc.delayed_status"). This historical AgcStatus structure contains the total exposure value calculated when the AGC sent the manual shutter/gain controls to the pipeline handler. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Tested-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-29pipeline: ipa: raspberrypi: Use IPA cookiesNaushir Patuck
Pass an IPA cookie from the pipeline handler to the IPA and eventually back to the pipeline handler through the setDelayedControls signal. This cookie is used to index the RPiController::Metadata object to be used for the frame. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Tested-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-29ipa: raspberrypi: Use an array of RPiController::Metadata objectsNaushir Patuck
Allow the IPA to cycle through an array of RPiController::Metadata objects, one per prepare()/process() invocation, when running the controller algorithms. This allows historical metadata objects to be retained, and subsequently passed into the controller algorithms on future frames. At present, only a single index into this array is used. This change provides a route to fixing a problem with the AGC algorithm, where if a manual shutter/gain is requested, the algorithm does not currently retain any context of the total exposure that it has calculated. As a result, the wrong digital gain would be applied when the frame with the manual shutter/gain is processed by the ISP. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Tested-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-29ipa: raspberrypi: Add RPiController::Metadata::mergeCopyNaushir Patuck
Add a new member function to RPiController::Metadata that copies unique key/value pairs from one object to the other. This is different from std::map::merge that would remove the key/value pairs from the source object. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-25ipa: rkisp1: agc: Support raw captureLaurent Pinchart
Support raw capture by allowing manual control of the exposure time and analogue gain. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-11-25ipa: rkisp1: Add support for manual gain and exposurePaul Elder
Add support for manual gain and exposure in the rkisp1 IPA. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-11-25ipa: rkisp1: Support raw capture in IPA operationsLaurent Pinchart
The RkISP1 can capture raw frames by bypassing the ISP. In that mode, the ISP will not produce statistics nor consume parameters. Most algorithms will thus be unable to run, with one exception: the AGC will still be able to configure the sensor exposure time and analog gain in manual mode. To prepare for this, add the ability to disable algorithms for the duration of the capture session based on the camera configuration. Individual algorithms report whether they support raw formats at construction time, and the IPA module disables algorithms in configure() based on the stream configurations. Disabled algorithms are skipped during the capture session in the processStatsBuffer() operation. As the ISP doesn't produce statistics, don't try to access the stats buffer. There is no need for similar logic in fillParamsBuffer() as that operation won't be called for raw capture. All algorithms report not supporting raw capture by default. Raw support in AGC will be added separately. The feature is implemented in the RkISP1 module without any support from libipa at this point to avoid designing a generic API based on a single user. This may be changed in the future. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-11-23ipa: rkisp1: add FrameDurationLimits controlNicholas Roth
Currently, the Android HAL does not work on rkisp1-based devices because required FrameDurationLimits metadata is missing from the IPA implementation. This change sets FrameDurationLimits for rkisp1 based on the existing ipu3 implementation, using the sensor's reported range of vertical blanking intervals with the minimum reported horizontal blanking interval. Signed-off-by: Nicholas Roth <nicholas@rothemail.net> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-23ipa: rkisp1: Fail hard on empty CameraSensorInfoJacopo Mondi
The RkISP1 pipeline and IPA module allows for the CameraSensorInfo to be empty, probably to accommodate some sensor used in a test platform that does not provide the mandatory libcamera requirements. As the \todo item in the IPA reports, there is a possibility that the received CameraSensorInfo is empty and it should be checked before accessing it, but currently such requirement is not enforced in the code. This allows to assume all the test platforms in use have now successfully moved their sensor driver to comply with the minimum requirements and provide a populated CameraSensorInfo to the IPA. As the safety check is not enforced, and as we don't want to allow faulty sensors to send empty CameraSensorInfo to the IPA, remove the \todo item in the IPA and fail hard in the pipeline handler if the sensor does not comply with libcamera requirements. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-23ipa: rkisp1: Use IPAConfig in IPA::configure()Jacopo Mondi
The RkISP1 implementation of IPA::configure() still uses the legacy interface where sensor controls (and eventually lens controls) are passed from the pipeline handler to the IPA in a map. Since the introduction of mojom-based IPA interface definition, it is possible to define custom data types and use them in the interface definition between the pipeline handler and the IPA. Align the RkISP1 IPA::configure() implementation with the one in the IPU3 IPA module by using a custom data type instead of relying on a map to pass controls to the IPA. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-23libcamera: Move IPA sensor controls validation to CameraSensorJacopo Mondi
The CameraSensor class validates that the sensor driver in use supports the controls required for IPA modules to work correctly. For in-tree IPA modules, whose pipeline handlers already use CameraSensor there's no need to validate such controls again. Remove controls validation from the IPU3 and RkISP1 IPA modules and rely on CameraSensor doing that at initialization time. The list of mandatory controls is expanded to add V4L2_CID_ANALOGUE_GAIN without which IPA modules cannot function. The new requirement only applies to RAW sensors, platforms like UVC and Simple are not impacted by this change. While at it, expand the sensor driver requirements documentation to include V4L2_ANALOGUE_GAIN in the list of mandatory controls a sensor driver has to support. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-23pipeline: raspberrypi: Remove enum BuffferMask from the mojom interfaceNaushir Patuck
The BufferMask enum provides a way of identifying which stream a frame buffer belongs to. This enum is defined in the raspberrypi.mojom interface file. However, the IPA does not need these enum definitions to mmap buffers that it uses. Move this enum out of the raspberrypi.mojom interface file and put it into the RPi namespace visible only to the pipeline handler. This removes the need to include the auto-generated IPA interface header in the RPi::Stream definition. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-21ipa: raspberrypi: Remove generic "pause" mechanism from AlgorithmNick Hollinghurst
No existing Algorithm used the base pause(), resume() functions or the paused_ flag, nor is there a need for a generic pause API. Remove these. The AGC and AWB algorithms now have methods named disableAuto(), enableAuto() which better describe their functionality. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-12ipa: rkisp1: Add rudimentary LSC tables for imx219Paul Elder
Add rudimentary LSC tables for imx219. These were generated with the rkisp1 tuning script from libtuning [1], using an imx8mp (debix) and a white computer monitor, at only a single color temperature of 5800. [1] https://lists.libcamera.org/pipermail/libcamera-devel/2022-October/035017.html Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-11-12ipa: rkisp1: Take into account color temperature during LSC algorithmFlorian Sylvestre
Add coefficients sets in the YAML tuning file to allow using different set depending of the image color temperature (provided by AWB algorithm). During processing, LSC algorithm computes coefficients by doing a linear interpolation between the two closer set. Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com> Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-11-09ipa: rkisp1: Move shutter speed and analogue gain limits from agc to sensorLaurent Pinchart
The limits for the shutter speed and analogue gain are stored in IPASessionConfiguration::agc. While they're related to the AGC, they are properties of the sensor, and are stored in the session configuration by the IPA module, not the AGC algorithm. Move them to the IPASessionConfiguration::sensor structure where they belong. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-11-09ipa: Sort algorithm operations based on calling orderLaurent Pinchart
Reorder functions in the base ipa::Algorithm and its derived classes to match the calling order: queueRequest(), prepare() and process(). This makes the code flow easier to read. No functional change intended. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-11-03ipa: rkisp1: Compute LSC algorithm parameter during configureFlorian Sylvestre
LSC gradient parameters are currently computed during prepare() phase. Because these parameters can be computed only one time and stay constant for each frame after, move the computation to the configure() function. Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
2022-10-28ipa: raspberrypi: awb: Update colour temperature whenever manual gains changeDavid Plowman
Previously we only did this when the system starts (on the first switch_mode). Now we do it whenever the manual colour gains are updated. To facilitate this, this R/B vs. colour temperature inverse functions are stored persistently in the AwbConfig. Signed-off-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>