libcamera/vivid.git - libcamera pipeline handler for VIVID

Age	Commit message (Collapse)	Author
2021-10-15	libcamera: ipu3: Rationalize constant expressions names	Jacopo Mondi
	Following the previous patch that moved all the ImgU-related contants in the ImgUDevice class namespace and that aligned their naming scheme to the 'kNameOfConstant' scheme, apply the same changes to the other components of the IPU3 pipeline handler. Cosmetic change, no functional changes intended. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kiera<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-battery"><rect x="1" y="6" width="18" height="12" rx="2" ry="2"></rect><line x1="23" y1="13" x2="23" y2="11"></line></svg>

e can remove the workaround and select the most opportune sensor size to feed the ImgU with. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-15libcamera: v4l2_subdevice: Set format field to V4L2_FIELD_NONELaurent Pinchart libcamera doesn't support interlaced formats, set the field to V4L2_FIELD_NONE explicitly instead of relying on drivers to interpret V4L2_FIELD_ANY the way we want it. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15libcamera: media_device: Print link information when setup failsLaurent Pinchart When setting up a link fails, the error message doesn't specify which link is being acted on. This makes debugging more difficult than it should be. Improve the message by printing the link information. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15lc-compliance: Remove using namespace in header filesHirokazu Honda "using namespace" in a header file propagates the namespace to the files including the header file. So it should be avoided. This removes "using namespace" in header files in lc-compliance. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-15v4l2: Remove using namespace in header filesHirokazu Honda "using namespace" in a header file propagates the namespace to the files including the header file. So it should be avoided. This removes "using namespace" in header files in v4l2. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-15qcam: Remove using namespace in header filesHirokazu Honda "using namespace" in a header file propagates the namespace to the files including the header file. So it should be avoided. This removes "using namespace" in header files in qcam. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-15cam: Remove using namespace in stream_options.hHirokazu Honda "using namespace" in a header file propagates the namespace to the files including the header file. So it should be avoided. This removes "using namespace" in stream_options.h Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-15ipa: raspberrypi: agc: Remove using namespace in agc.hppHirokazu Honda "using namespace" in a header file propagates the namespace to the files including the header file. So it should be avoided. This removes "using namespace" in agc.hpp. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-15cam: Drop frames once the capture limit is reachedLaurent Pinchart The camera session keeps requeuing requests until the capture limit is reached. This causes more request than the limit to complete, as there's a queue of requests in flight. When capturing from multiple cameras concurrently, this results in the captureDone signal being emitted for every request completion after the limit is reached, instead of once per camera session when reaching the limit. Fix this by simply dropping any request that completes after the limit is reached. We could instead avoid requeuing more requests than needed to reach the limit, but that may cause request starvation in pipelines, which are currently not handled consistently (or correctly). Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15cam: Pass stream names to FileSinkLaurent Pinchart The FileSink class constructs stream names internally the same way that the CameraSession does, except that it fails to add the camera name. This results in files being written without the camera name. This could be fixed in FileSink, but we would still duplicate code to construct stream names. Pass the stream names map from CameraSession to FileSink instead, and store it internally. Fixes: 02001fecb0f5 ("cam: Turn BufferWriter into a FrameSink") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15cam: Rename CameraSession::streamName_ to streamNames_Laurent Pinchart The streamName_ map contains names for all streams, rename it to streamNames_ to make this more explicit. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15libcamera: base: backtrace: Fallback to libunwind for symbolic namesLaurent Pinchart libunwind has an API to provide symbolic names for functions. It's less optimal than using backtrace_symbols() or libdw, as it doesn't allow deferring the symbolic names lookup, but it can be usefull as a fallback if no other option is available. A sample backtrace when falling back to libunwind looks like libcamera::VimcCameraData::init()+0xbd libcamera::PipelineHandlerVimc::match(libcamera::DeviceEnumerator*)+0x3e0 libcamera::CameraManager::Private::createPipelineHandlers()+0x1a7 libcamera::CameraManager::Private::init()+0x98 libcamera::CameraManager::Private::run()+0x9f libcamera::Thread::startThread()+0xee decltype(*(std::__1::forward<libcamera::Thread*>(fp0)).*fp()) std::__1::__invoke<void (libcamera::Thread::*)(), libcamera::Thread*, void>(void (libcamera::Thread::*&&)(), libcamera::Thread*&&)+0x77 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (libcamera::Thread::*)(), libcamera::Thread*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (libcamera::Thread::*)(), libcamera::Thread*>&, std::__1::__tuple_indices<2ul>)+0x3e void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (libcamera::Thread::*)(), libcamera::Thread*> >(void*)+0x62 start_thread+0xde ??? Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15libcamera: base: backtrace: Use libunwind when availableLaurent Pinchart libunwind is an alternative to glibc's backtrace() to extract a backtrace. Use it when available to extend backtrace support to more platforms. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15libcamera: base: backtrace: Use libdw to provide symbolic namesLaurent Pinchart libdw provides access to debugging information. This allows creating better stack trace entries, with file names and line numbers, but also with demangled symbols as the symbol name is available and can be passed to abi::__cxa_demangle(). With libdw, the backtrace previously generated by backtrace_symbols() src/cam/../libcamera/libcamera.so(_ZN9libcamera14VimcCameraData4initEv+0xbd) [0x7f7dbb73222d] src/cam/../libcamera/libcamera.so(_ZN9libcamera19PipelineHandlerVimc5matchEPNS_16DeviceEnumeratorE+0x3e0) [0x7f7dbb731c40] src/cam/../libcamera/libcamera.so(_ZN9libcamera13CameraManager7Private22createPipelineHandlersEv+0x1a7) [0x7f7dbb5ea027] src/cam/../libcamera/libcamera.so(_ZN9libcamera13CameraManager7Private4initEv+0x98) [0x7f7dbb5e9dc8] src/cam/../libcamera/libcamera.so(_ZN9libcamera13CameraManager7Private3runEv+0x9f) [0x7f7dbb5e9c5f] src/cam/../libcamera/base/libcamera-base.so(_ZN9libcamera6Thread11startThreadEv+0xee) [0x7f7dbb3e95be] src/cam/../libcamera/base/libcamera-base.so(+0x4f9d7) [0x7f7dbb3ec9d7] src/cam/../libcamera/base/libcamera-base.so(+0x4f90e) [0x7f7dbb3ec90e] src/cam/../libcamera/base/libcamera-base.so(+0x4f2c2) [0x7f7dbb3ec2c2] /lib64/libpthread.so.0(+0x7e8e) [0x7f7dbab65e8e] /lib64/libc.so.6(clone+0x3f) [0x7f7dbb10b26f] becomes libcamera::VimcCameraData::init()+0xbd (src/libcamera/libcamera.so [0x00007f9de605b22d]) libcamera::PipelineHandlerVimc::match(libcamera::DeviceEnumerator*)+0x3e0 (src/libcamera/libcamera.so [0x00007f9de605ac40]) libcamera::CameraManager::Private::createPipelineHandlers()+0x1a7 (src/libcamera/libcamera.so [0x00007f9de5f13027]) libcamera::CameraManager::Private::init()+0x98 (src/libcamera/libcamera.so [0x00007f9de5f12dc8]) libcamera::CameraManager::Private::run()+0x9f (src/libcamera/libcamera.so [0x00007f9de5f12c5f]) libcamera::Thread::startThread()+0xee (src/libcamera/base/libcamera-base.so [0x00007f9de5d125be]) decltype(*(std::__1::forward<libcamera::Thread*>(fp0)).*fp()) std::__1::__invoke<void (libcamera::Thread::*)(), libcamera::Thread*, void>(void (libcamera::Thread::*&&)(), libcamera::Thread*&&)+0x77 (src/libcamera/base/libcamera-base.so [0x00007f9de5d159d7]) void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (libcamera::Thread::*)(), libcamera::Thread*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (libcamera::Thread::*)(), libcamera::Thread*>&, std::__1::__tuple_indices<2ul>)+0x3e (src/libcamera/base/libcamera-base.so [0x00007f9de5d1590e]) void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (libcamera::Thread::*)(), libcamera::Thread*> >(void*)+0x62 (src/libcamera/base/libcamera-base.so [0x00007f9de5d152c2]) start_thread+0xde (/var/tmp/portage/sys-libs/glibc-2.33-r1/work/glibc-2.33/nptl/pthread_create.c:482) __clone+0x3f (../sysdeps/unix/sysv/linux/x86_64/clone.S:97) The stack entries related to libcamera are missing source file name and line information, which will be investigated separately, but this is still an improvement. Use libdw when available, falling back to backtrace_symbols() otherwise, or if libdw fails for any reason. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15libcamera: base: Add Backtrace classLaurent Pinchart Create a new class to abstract generation and access to call stack backtraces. The current implementation depends on the glibc backtrace() implementation and is copied from the logger. Future development will bring support for libunwind, transparently for the users of the class. The logger backtrace implementation is dropped, replaced by usage of the new Backtrace class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-15libcamera: pipeline: ipu3: Use new Size grownBy() and shrunkBy() helpersLaurent Pinchart The Size class has new helpers that can simplify the code in the IPU3 pipeline handler. Use them. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-10-15libcamera: geometry: Add Size members to grown or shrink by a marginLaurent Pinchart Add four new member functions to the Size class (two in-place and two const) to grow and shrink a Size by given margins. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-10-13android: Check if Stream configurations were generated correctlyJavier Martinez Canillas The libcamera Android Camera HAL generates camera configurations for the StillCapture, Raw and ViewFinder stream roles. But there is only a check if the configuration generation failed, for the StillCapture stream role. This could lead to a NULL pointer dereference if a pipeline handler fails to generate a default configuration for one of the other two stream roles. Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-12android: camera_device: Fix crash in calling CameraDevice::close()Hirokazu Honda The problem is happening because we seem to add a CameraStream associated buffer (depending on the CameraStream::Type) to the Request, in CameraDevice::processCaptureRequest(). However, when the camera stops, all the current buffers are marked with FrameMetadata::FrameCancelled and proceed to completion. But the buffer associated with the CameraStream (that was previously added to the request) has now been cleared out with a part of streams_.clear(), even before the camera stop() has been invoked. Any access to those request buffers after they have been cleared, will result in a crash. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> 2021-10-10android: camera_stream: Fix error message for buffer creationLaurent Pinchart Creating a CameraBuffer instance doesn't map memory. Fix the error message accordingly. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> 2021-10-07pipeline: raspberrypi: Create empty control lists correctlyDavid Plowman When the pipeline handler start() method is supplied with a NULL list of controls, we send an empty control list to the IPA. When the IPA is running in isolated mode the control list goes through the data serializer, for which it must be marked correctly as a list of "controls::controls", otherwise the IPA process will abort. The IPA has a similar problem returning a control list in its configure() method. We must be careful to initialise it properly even when empty. Signed-off-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: Replace ipa::ipu3::algorithms::Ipu3AwbCellJean-Michel Hautbois The intel-ipu3.h public interface from the kernel does not define how to parse the statistics for a cell. This had to be identified by a process of reverse engineering, and later identifying the structures from [0] leading to our custom definition of struct Ipu3AwbCell. [0] https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h To improve the kernel interface, a proposal has been made to the linux-kernel [1] to incorporate the memory layout for each cell into the intel-ipu3 header directly. [1] https://lore.kernel.org/linux-media/20211005202019.253353-1-jeanmichel.hautbois@ideasonboard.com/ Update our local copy of the intel-ipu3.h to match the proposal and change the AGC and AWB algorithms to reference that structure directly, allowing us to remove the deprecated custom Ipu3AwbCell definition. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: agc: Rewrite and simplify the brightness loopJean-Michel Hautbois Now that we know how the AWB statistics are formatted, use a simplified loop in processBrightness() to parse the green values and get the histogram. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: awb: Introduce Black Level CorrectionJean-Michel Hautbois The pixels output by the camera normally include a black level, because sensors do not always report a signal level of '0' for black. Pixels at or below this level should be considered black and to achieve that, we need to substract an offset to all the pixels. This can be taken into account by reading the lowest value of a special region on sensors which is not exposed to light. This provides a substracting factor to be able to adjust the expected black levels in the resulting images. For a camera outputting 10-bit pixel values (in the range 0 to 1023) a typical black level might be 64. It is a fixed value, obtained by capturing a raw frame with minimum exposure and gain fixed to 1.0 while covering the sensor (the darker the better). We consider it good enough as a very first approximation, until we measure it during a tuning process and include it in a configuration file Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: awb: Use the line stride for the statsJean-Michel Hautbois The statistics buffer 'ipu3_uapi_awb_raw_buffer' stores the ImgU calculation results in a buffer aligned horizontally to a multiple of 4 cells. The AWB loop should take care of it to add the proper offset between lines and avoid any staircase effect. It is no longer required to pass the grid configuration context to the private functions called from process() which simplifies the code flow. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: awb: Correct the gain multipliersJean-Michel Hautbois The gains have a precision u3.13, range [0, 8[ which means that a gain multiplier value of 1.0 is represented as a multiplication by 8192 in the ImgU. Correct the gains as this was misunderstood in the first place. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: awb: Correct the relevant zones proportionJean-Michel Hautbois The algorithm uses the statistics of a cell only if there is not too much saturated pixels in it. The grey world algorithm works fine when there are a limited number of outliers. Consider a valid zone to be at least 80% of unsaturated cells in it. This value could very well be configurable, and make the algorithm more or less tolerant. While at it, implement it in a configure() call as it will not change during execution, and cache the cellsPerZone values estimated with std::round as we are using cmath. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: Change limits and split loops in calculateBdsGrid()Jean-Michel Hautbois The loops over the width and height of the image when calculating the BDS grid parameters are nested, but they're actually independent. Split them to reduce the complexity. While at it, split out the constants to documented const expressions for the grid sizes. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: Change the limits of the AWB statsJean-Michel Hautbois Until now, the limits used to calculate the grid based on the Bayer Down Scaler configuration where taken from the kernel documentation [0]. While testing and understanding the format of the ImgU statistics, it appears that the ones defined in CrOS [1] are the correct ones. Use those. [0] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-meta-intel-ipu3.html?highlight=v4l2_meta_fmt_ipu3_params#intel-ipu3-imgu-uapi-data-types [1] https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: awb: Make the naming consistentJean-Michel Hautbois The variables mix the terms cell, region and zone. It can confuse the reader, and make the algorithm more difficult to follow. Rename the local variables to be consistent with their definitions: - Cells are defined in Pixels - Zones are defined in Cells There is no "region" as such, so replace it with the correct term. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: Change Accumulator structure layoutJean-Michel Hautbois The pixel component sums for the Accumulator are inconsistent with other similar structures such as the IPAFrameContext::awb::gains. Group the red, green, and blue sums together in a struct and store them as uint64_t to reduce potential architectural differences. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: Rename IspStatsRegion to AccumulatorJean-Michel Hautbois The IspStatsRegion structure was introduced as an attempt to prepare for a generic AWB algorithm structure. The structure name by itself is not explicit and it is too optimistic to try and make a generic one for now. Its role is to accumulate the pixels in a given zone. Rename it to accumulator, and remove the uncounted field at the same time. It is always possible to know how many pixels are not relevant for the algorithm by calculating total-counted. The uncounted field was only declared and not used. Amend the documentation accordingly. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06ipa: ipu3: Move the AWB stats structuresJean-Michel Hautbois The structure Ipu3AwbCell describes the AWB stats layout on the kernel side. We will need it to be used by the AGC algorithm to be introduced later, so let's make it visible from ipa::ipu3::algorithms and not only for the AWB class. The IspStatsRegion will be needed by AGC too, so let's move it in the same namespace too. Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-06android: camera_stream: Set right format for processor output bufferHirokazu Honda CameraStream always sets the format of processor output buffer to MJPEG. This fixes the issue. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> 2021-10-05libcamera: Fix crash caused by reading uninitialised delayed controlsDavid Plowman The cause is that we read out delayed values using a frame's sequence number (DelayedControls::get). But we fill the values up (DelayedControls::applyControls) incrementing writeCount by only one even if the sequence number has jumped by several since last time. This is exactly what happens when frames are being dropped. So the fix is to increment writeCount by "as much as the sequence number has jumped since last time", which means that we just follow the sequence number directly. Bug: https://bugs.libcamera.org/show_bug.cgi?id=74 Signed-off-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Tested-by: Naushir Patuck <naush@raspberrypi.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-05cam: kms_sink: Print display pipeline configurationLaurent Pinchart It can be useful, for diagnosis purpose, to know what plane and CRTC the KMS sink auto-selects. Print the display pipeline configuration at start time. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> 2021-10-05libcamera: camera_sensor: Reverse the key and value of test pattern mode mapHirokazu Honda The key and value of the test pattern mode are originally the index of v4l2 control and the corresponding test pattern mode control value. This key and value are useful in the initialization for reporting available test pattern modes. However, the map of the reversed key and value is much more useful in applying a requested test pattern mode. Reverses the key and value of the map as the initialization is one time but the test pattern mode request will be multiple times. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-10-05gstreamer: Check if Stream configurations were generated correctlyJavier Martinez Canillas Currently, the gst_libcamera_device_new() function assumes that a call to Camera::generateConfiguration() will always succeed, but that may not be the case and the return value must to be checked. Otherwise, this could lead to a NULL pointer dereference if the pipeline handler fails to generate a config for the VideoRecording stream role. Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-09-30android: camera_device: Send capture results by inspecting the queueUmang Jain There is a possibility that an out-of-order completion of capture request happens by calling process_capture_result() directly on error paths. The framework expects that errors should be notified as soon as possible, but the request completion order should remain intact. An existing instance of this is abortRequest(), which sends the capture results on flushing state, without considering order-of-completion. Since we have a queue of Camera3RequestDescriptor tracking each capture request placed by framework to libcamera HAL, we should be only sending back capture results from a single location, by inspecting the queue. As per the patch, this now happens in CameraDevice::sendCaptureResults(). Each descriptor is now equipped with its own status to denote whether the capture request is complete and ready to be send back to the framework or needs to be waited upon. This ensures that the order of completion is respected for the requests. Since we are fixing out-of-order request completion in abortRequest(), change the function to read from the Camera3RequestDescriptor directly, instead of camera3_capture_request_t. The descriptor should have all the information necessary to set the request buffers' state to error. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-09-30android: camera_device: Mark abortRequest() and notifyError() as constUmang Jain abortRequest() and notifyError() do not modify any members of CameraDevice hence, these functions can be const. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-09-30android: camera_device: Transform descriptors_ map to queueUmang Jain The descriptors_ map holds Camera3RequestDescriptor(s) which are per-capture requests placed by the framework to libcamera HAL. CameraDevice::requestComplete() looks for the descriptor for which the camera request has been completed and removes it from the map. Since the requests are placed in form of FIFO and the framework expects the order of completion to be FIFO as well, this calls for a need of a queue rather than a std::map. This patch still keeps the same lifetime of Camera3RequestDescriptor as before i.e. in the requestComplete(). Previously, a descriptor was extracted from the map and its lifetime was bound to requestComplete(). The lifetime is kept the same by manually calling .pop_front() on the queue. In the subsequent commit, this is likely to change with a centralized location of dropping descriptors from the queue for request completion. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-09-30android: camera_worker: Use Camera3RequestDescriptor as cookieUmang Jain Use Camera3RequestDescriptor as cookie for the Capture Request. The cookie is used to lookup descriptors map in CameraDevice::requestComplete(). The map will be transformed to a queue in subsequent commit. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-09-29android: Wait on fences in CameraStream::process()Jacopo Mondi Acquire fences for streams of type Mapped generated by post-processing are not correctly handled and are currently ignored by the camera HAL. Fix this by adding CameraStream::waitFence(), executed before starting the post-processing in CameraStream::process(). The change applies to all streams generated by post-processing (Mapped and Internal) but currently acquire fences of Internal streams are handled by the camera worker. Postpone that to post-processing time by passing -1 to the Worker for Internal streams. Also correct the release_fence handling for failed captures, as the framework requires the release fences to be set to the acquire fence value if the acquire fence has not been waited on. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Tested-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-09-28meson: Set a SONAME version in the libcamera librariesJavier Martinez Canillas The libcamera and libcamera-base libraries are currently unversioned, but donwstream users expect these to have a proper SONAME version in order to package them. A stable release has not yet happened because the project is still under development and the API/ABI might change. But having a versioned SONAME would allow distributions to package libcamera, without the need to add any downstream patch to set the version. Since the "0.0.0" version is already used in different places, let's also use that as the library version. The meson build system will use the first part of the version ("0") as the SONAME version, which is aligned with the convention used by other projects: $ ls /lib64/libcamera*so* -1 /lib64/libcamera-base.so /lib64/libcamera-base.so.0 /lib64/libcamera-base.so.0.0.0 /lib64/libcamera.so /lib64/libcamera.so.0 /lib64/libcamera.so.0.0.0 $ objdump -p /lib64/libcamera.so.0.0.0 | grep SONAME SONAME libcamera.so.0 Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-09-28android: camera_device: Fix race on queuing capture requestUmang Jain The Camera3RequestDescriptor containing the capture request is added to the descriptors_ map after a call to CameraWorker::queueRequest(). This is a race condition since CameraWorker::queueRequest() queues requests to libcamera::Camera asynchronously. The requests may thus complete before they get added to descriptors_, in which case requestComplete() will fail to lookup the request in the map. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> 2021-09-28gstreamer: Fix spelling of the work manager used in a util functionVedant Paranjape Fix all name in all instances of the function gst_libcamera_get_camera_mananger to gst_libcamera_get_camera_manager. Spelling of manager was incorrect. This patch has no functional changes. Signed-off-by: Vedant Paranjape <vedantparanjape160201@gmail.com> Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 2021-09-28libcamera: control_serializer: Fix usage of uninitialized variableLaurent Pinchart The idMap variable may be used uninitialized in the ControlSerializer::deserialize<ControlList>() function as reported by gcc 11: ../../src/libcamera/control_serializer.cpp: In member function ‘T libcamera::ControlSerializer::deserialize(libcamera::ByteStreamBuffer&) [with T = libcamera::ControlList]’: ../../src/libcamera/control_serializer.cpp:609:33: error: ‘idMap’ may be used uninitialized in this function [-Werror=maybe-uninitialized] 609 | ControlList ctrls(*idMap); | This is due to a missing default case in a switch/case. Fix it by adding the default case. Fixes: 6b1404fc4836 ("libcamera: control_serializer: Fix usage of uninitialized variable") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> 2021-09-27android: Fix generation of thumbnail for EXIF dataUmang Jain Generation of thumbnail is not occuring currently because ANDROID_JPEG_THUMBNAIL_SIZE is not set for request metadata passed to PostProcessorJpeg::process(). The commit 1264628d3c92("android: jpeg: Configure thumbnailer based on request metadata") introduced the mechanism to retrieve the thumbanil size from request metadata, however it didn't add the counterpart i.e. inserting the size in the request metadata in request metadata template, at the first place. The patch fixes this issue by setting ANDROID_JPEG_THUMBNAIL_SIZE in the request metadata template populated by CameraCapabilities::requestTemplatePreview(). The value for ANDROID_JPEG_THUMBNAIL_SIZE is set to be the first non-zero size reported by static metadata ANDROID_JPEG_AVAILABLE_THUMBNAIL_SIZES. Fixes: 1264628d3c92("android: jpeg: Configure thumbnailer based on request metadata") Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>