summaryrefslogtreecommitdiff
path: root/src/libcamera/pipeline
AgeCommit message (Collapse)Author
2021-11-11libcamera: pipeline: ipu3: Fix minCrop indentationKieran Bingham
When updating the controls the calculation for minCrop incorrectly indents the parameters to scaledBy(). Fix it. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-02pipeline: raspberrypi: Fix rounding issue in findBestFormat()Naushir Patuck
The aspect ratio calculation divides two integer values then casts to a double. This might reduce precision when scoring for aspect rato differences. Fix this by casting the integer to a double before the division. Reported-by: Coverity CID=361652 Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02pipeline: raspberrypi: Account for a missing Unicam embedded data nodeNaushir Patuck
The unicam driver no longer registers an embedded data node if the sensor does not provide this stream. Account for this in the pipeline handler match routine by not assuming it is always present. Add a warning if Unicam and the CamHelper do not agree on the presense of sensor embedded data, and disable its usage in these cases. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02pipeline: raspberrypi: Remove "unpacked" format penalty in mode selectionNaushir Patuck
With the recent change to switch to programming the sensor device directly, the notion of packed vs unpacked modes are not relevent, since that is a Unicam format construct. Remove any scoring based on packed/unpacked modes. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02pipeline: raspberrypi: Convert the pipeline handler to use media controllerNaushir Patuck
Switch the pipeline handler to use the new Unicam media controller based driver. With this change, we directly talk to the sensor device driver to set controls and set/get formats in the pipeline handler. This change requires the accompanying Raspberry Pi linux kernel change at https://github.com/raspberrypi/linux/pull/4645. If this kernel change is not present, the pipeline handler will fail to run with an error message informing the user to update the kernel build. Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02pipeline: raspberrypi: Apply sensor flips at the start of configure()Naushir Patuck
Sensor flips might change the Bayer order of the requested format. The existing code would set a sensor format along with the appropriate Unicam and ISP input formats, but reset the latter two on start() once the flips had been requested. We can now set the sensor flips just before we set the sensor mode in configure(), thereby not needing the second pair of format sets in start(). Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-15ipa: ipu3: Update camera controls in configure()Jacopo Mondi
When a new CameraConfiguration is applied to the Camera the IPA is configured as well, using the newly applied sensor configuration and its updated V4L2 controls. Also update the Camera controls at IPA::configure() time by re-computing the controls::ExposureTime and controls::FrameDurationLimits limits and update the controls on the pipeline handler side after having configured the IPA. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15libcamera: ipu3: Split controls init/updateJacopo Mondi
In order to prepare for updating the Camera controls limits when a new camera configuration is applied, split the initControls() function in two: - updateControls() to actually compute controls values - initControls() to initialize the sensor configuration and call updateControls Update the functions documentation accordingly. No functional changes intended. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-15libcamera: ipu3: Rationalize constant expressions namesJacopo Mondi
Following the previous patch that moved all the ImgU-related contants in the ImgUDevice class namespace and that aligned their naming scheme to the 'kNameOfConstant' scheme, apply the same changes to the other components of the IPU3 pipeline handler. Cosmetic change, no functional changes intended. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-15libcamera: ipu3: Centralize ImgU sizes definitionJacopo Mondi
The definition of several constants that describe the ImgU characteristics are spread between two files: ipu3.cpp and imgu.cpp. As the ipu3.cpp uses definitions from the imgu.cpp file, in order to remove the usage of magic numbers, it is required to move the definitions to a common header file where they are accessible to the other .cpp modules. Move all the definitions of the ImgU sizes and alignments to the ImgUDevice class as static constexpr and update their users accordingly. Cosmetic changes, no functional changes intended. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15libcamera: ipu3: Use the optimal sensor sizeJacopo Mondi
As reported by commit 7208e70211a6 ("libcamera: ipu3: Always use sensor full frame size") the current implementation of the IPU3 pipeline handler always uses the sensor resolution as the ImgU input frame size in order to work around an issue with the ImgU configuration procedure. Now that the frame selection policy has been modified in the CIO2Device class implementation to comply with the requirements of the ImgU configuration script we can remove the workaround and select the most opportune sensor size to feed the ImgU with. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15libcamera: pipeline: ipu3: Use new Size grownBy() and shrunkBy() helpersLaurent Pinchart
The Size class has new helpers that can simplify the code in the IPU3 pipeline handler. Use them. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-10-07pipeline: raspberrypi: Create empty control lists correctlyDavid Plowman
When the pipeline handler start() method is supplied with a NULL list of controls, we send an empty control list to the IPA. When the IPA is running in isolated mode the control list goes through the data serializer, for which it must be marked correctly as a list of "controls::controls", otherwise the IPA process will abort. The IPA has a similar problem returning a control list in its configure() method. We must be careful to initialise it properly even when empty. Signed-off-by: David Plowman <david.plowman@raspberrypi.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-09-27libcamera: ipu3: Drop entityControls mapJacopo Mondi
The IPA::configure() function has an IPAConfigInfo parameters which contains a map of numerical indexes to ControlInfoMap instances. This is a leftover of the old IPA protocol, where it was not possible to specify a rich interface as it is possible today and each entity ControlInfoMap was indexed by a numerical id and stored in a map. Now that the IPA interface allows to specify parameters by name, drop the map and send the sensor's control info map only. If we'll need more ControlInfoMap to be shared with the IPA, a new parameter can be added to IPAConfigInfo. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-09-07libcamera: Use V4L2PixelFormat::fromPixelFormat()Laurent Pinchart
Replace manual searches for V4L2 pixel format in the PixelFormatInfo with the V4L2PixelFormat::fromPixelFormat() helper function. This prepares for multi-planar support that will modify how V4L2 pixel formats are stored in PixelFormatInfo. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-09-07libcamera: v4l2_videodevice: Drop toV4L2PixelFormat()Laurent Pinchart
The V4L2VideoDevice::toV4L2PixelFormat() function is incorrectly implemented, as it will pick a multi-planar format if the device supports the multi-planar API, even if only single-planar formats are supported. This currently works because the implementation calls V4L2PixelFormat::fromPixelFormat(), which ignores the multiplanar argument and always returns a single-planar format. Fixing this isn't trivial. As we don't need to support multi-planar V4L2 formats at this point, drop the function instead of pretending everything is fine, and call V4L2PixelFormat::fromPixelFormat() directly from pipeline handlers. As the single-planar case is the most common, set the multiplanar argument to false by default to avoid long lines. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-08-31libcamera: pipeline: simple: Remove SimplePipelineHandler::activeCamera_Laurent Pinchart
The SimplePipelineHandler activeCamera_ member pointer is set but never used. Drop it. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Move bufferReady handler to SimpleCameraDataLaurent Pinchart
To use multiple cameras at the same time, a per-camera buffer ready handler is needed. Move the bufferReady() function connected to the V4L2VideoDevice bufferReady signal from the SimplePipelineHandler class to the SimpleCameraData class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Move converter to SimpleCameraDataLaurent Pinchart
To use multiple cameras at the same time, each camera instance will need its own converter. Store the converter in SimpleCameraData, and open it in init(). Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Add pipeline pad reservation mechanismLaurent Pinchart
The cameras created by the same pipeline handler instance may share hardware resources, prohibiting usage of multiple cameras concurrently. Implement a heuristic to reserve resources and handle mutual exclusiong in a generic way. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Open all video devices at match() timeLaurent Pinchart
Move opening of video devices at match() time, the same way as subdevs are opened, to make the handling of V4L2 video devices and subdevices more consistent. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Store all entity devices in common mapLaurent Pinchart
Merge the SimplePipelineHandler videos_ and subdevs_ maps, which respectively store V4L2 video devices and subdevices associated with entities, into a single entities_ map that contains an EntityData structure. This gathers all data about entities in a single place, allowing for easy extension of entity data in the future. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Store video node entity in camera dataLaurent Pinchart
Store the entity corresponding to the video node at the end of the pipeline in the SimpleCameraData::entities_ list. This requires special handling of the video node in the loops that iterate over all entities, but will be useful to implement mutually exclusive access to entities for concurrent camera usage. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Delay opening of video device until init()Laurent Pinchart
The video device is currently opened in the SimpleCameraData constructor. This requires opening the video devices on-demand the first time they're accessed, which gets in the way of refactoring of per-entity data storage in the pipeline handler. Move opening of the video device to the SimpleCameraData::init() function. The on-demand open mechanism isn't touched yet, it will be refactored later. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31libcamera: pipeline: simple: Add sink and source pads to entity dataLaurent Pinchart
Record the sink and source pads through which an entity is traversed in the list of entities stored in the camera data. This prepares for implementing mutually exclusive access to entities between cameras. The debug message that displays detected pipelines now prints pads to describe the pipeline more precisely: [0:00:35.901275750] [260] DEBUG SimplePipeline simple.cpp:404 Found pipeline: [imx290 2-001a|0] -> [0|csis-32e40000.csi|1] -> [0|mxc_isi.0|1] -> [0|mxc_isi.0.capture] Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31ipu3: cio2: Tweak sensor size selection policyUmang Jain
Do not compare higher precision of the ratios, as it might lead to absurd selection of sensor size for a relatively low requested resolution size. For example: The imx258 driver supports the following sensor resolutions: - 4208x3118 = 1.349583066 - 2104x1560 = 1.348717949 - 1048x780 = 1.343589744 It can be inferred that, that the aspect ratio only differs by a small mantissa with each other. It does not makes sense to select a 4208x3118 for a requested size of say 640x480 or 1280x720, which is what is happening currently. ($) cam -c1 -swidth=640,height=480,role=raw - CIO2 configuration: 4208x3118-SGRBG10_IPU3 [*] In order to address this constraint, only compare the ratio with single precision to make a better decision on the sensor resolution policy selection. ($) cam -c1 -srole=raw,width=640,height=480 - CIO2 configuration: 1048x780-SGRBG10_IPU3 [*] [*] Please revert 0536a9aa7189("ipu3: Disallow raw only camera configuration") if the configuration is reported as invalid. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31ipu3: cio2: Change sensor size selection policyUmang Jain
The current implementation of getSensorFormat() prioritizes sensor sizes that match the output size FOV ratio. Modify the frame size selection procedure to prioritize resolutions with the same FOV as the sensor's native one, to avoid cropping in the sensor pixel array. For example: - on a Soraka device equipped with ov13858 as back sensor, with a native resolution of 4224x3136 (4:3), when requested to select the sensor output size to produce 1080p (16:9) a frame size of 2112x1188 (16:9) is selected causing the ImgU configuration procedure to fail. If a resolution with the same FOV as the sensor's native size, such as 2112x1568 (4:3), is selected the pipeline works correctly. Suggested-by:: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31ipu3: cio2: Replicate CameraSensor::getFormats() to a member functionUmang Jain
This prepares a base to introduce custom selection of sensor format based on platform(Soraka or Nautilus) constraints. The changes in selection policy will be introduced in a subsequent patch. The function is copied from CameraSensor::getFormat() in the IPU3 pipeline handler code to be later changed on top. The copy is not verbatim and has a minor change as follows: CameraSensor::getFormats() has access to a V4L2Subdevice::Formats internally and use it directly to iterate over supported camera sensor frame sizes. The copy is adapted to use the CameraSensor::sizes(mbusCode) instead, to enumerate over the supported frame sizes as per mbusCodesToPixelFormat map. No functional changes in this patch. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31libcamera: camera_sensor: Transform CameraSensor::sizes()Umang Jain
In CameraSensor, the mbusCodes() and sizes() accessor functions retrieves all the supported media bus codes and the supported sizes respectively. However, this is quite limiting since the caller probably isn't in a position to match which range of sizes are supported for a particular mbusCode. Hence, the caller is most likely interested to know about the sizes supported for a particular media bus code. This patch transforms the existing CameraSensor::sizes() to CameraSensor::sizes(mbuscode) to achieve that goal. The patch also transforms existing CIO2Device::sizes() in IPU3 pipeline handler to CIO2Device::sizes(PixelFormat) on a similar principle. The function is then plumbed to CameraSensor::sizes(mbusCode) to enumerate the per-format sizes as required in PipelineHandlerIPU3::generateConfiguration(). Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-17pipeline: ipu3: Skip recording timestamp for cancelled buffersUmang Jain
There is no point in recording sensor's timestamp when V4L2VideoDevice has marked the frame buffers with FrameMetadata::FrameCancelled (happens when the streams are stopped). The metadata is mostly invalid for cancelled buffers hence, setting timestamp on invalid metadata does not make sense (however some metadata can be considered valid for e.g. returning cause of failure through metadata on cancelled buffers). Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-17libcamera: pipeline: Cast to derived pipeline handler with helpersLaurent Pinchart
Replace manual static casts from the PipelineHandler pointer to a derived class pointer with helper functions in the camera data classes. This simplifies code accessing the pipeline from the camera data. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-17libcamera: pipeline_handler: Drop CameraData classLaurent Pinchart
The CameraData class isn't used anymore. Drop it. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17libcamera: pipeline: ipu3: Migrate to Camera::PrivateLaurent Pinchart
As part of the effort to remove the CameraData class, migrate the pipeline handler-specific camera data from CameraData to the Camera::Private class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17libcamera: pipeline: raspberrypi: Migrate to Camera::PrivateLaurent Pinchart
As part of the effort to remove the CameraData class, migrate the pipeline handler-specific camera data from CameraData to the Camera::Private class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17libcamera: pipeline: rkisp1: Migrate to Camera::PrivateLaurent Pinchart
As part of the effort to remove the CameraData class, migrate the pipeline handler-specific camera data from CameraData to the Camera::Private class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17libcamera: pipeline: vimc: Migrate to Camera::PrivateLaurent Pinchart
As part of the effort to remove the CameraData class, migrate the pipeline handler-specific camera data from CameraData to the Camera::Private class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16libcamera: pipeline: uvcvideo: Migrate to Camera::PrivateLaurent Pinchart
As part of the effort to remove the CameraData class, migrate the pipeline handler-specific camera data from CameraData to the Camera::Private class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16libcamera: pipeline: simple: Migrate to Camera::PrivateLaurent Pinchart
As part of the effort to remove the CameraData class, migrate the pipeline handler-specific camera data from CameraData to the Camera::Private class. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-16libcamera: pipeline_handler: Move CameraData members to Camera::PrivateLaurent Pinchart
With pipeline handlers now being able to subclass Camera::Private, start the migration from CameraData to Camera::Private by moving the members of the base CameraData class. The controlInfo_, properties_ and pipe_ members are duplicated for now, to allow migrating pipeline handlers one by one. The Camera::Private class is now properly documented, don't exclude it from documentation generation. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16libcamera: camera: Pass Private pointer to Camera constructorLaurent Pinchart
In order to allow subclassing Camera::Private in pipeline handlers, pass the pointer to the private data to the Camera constructor, and to the Camera::createCamera() function. The Camera::Private id_ and streams_ members now need to be initialized by the Camera constructor instead of the Camera::Private constructor, to allow storage of the streams in a pipeline handler-specific subclass of Camera::Private. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16pipeline: vimc: Force complete of request on cancelled buffersUmang Jain
When the stream is stopped, the V4L2VideoDevice sends back all the queued buffers with FrameMetadata::FrameCancelled status. It is the responsibility of the pipeline handler to handle these buffers with FrameMetadata::FrameCancelled. VIMC is currently missing this handling path. As the FrameMetadata::FrameCancelled is set when the stream is stopped, we can be sure that no more queued and re-use of request shall happen. Hence, cancel all the requests' buffers force a complete with completeBuffer(). The issue is caught by the gstreamer_single_stream_test.cpp running with vimc. During the check with meson built-in option '-Db_sanitize=address,undefined' it was observed: ==118003==ERROR: AddressSanitizer: heap-use-after-free on address 0x60e000037108 at pc 0x7f225160c9ac bp 0x7f224a47b620 sp 0x7f224a47b618 READ of size 4 at 0x60e000037108 thread T1 #0 0x7f225160c9ab in libcamera::Request::sequence() const ../include/libcamera/request.h:55 #1 0x7f22518297aa in libcamera::VimcCameraData::bufferReady(libcamera::FrameBuffer*) ../src/libcamera/pipeline/vimc/vimc.cpp:577 #2 0x7f225183b1ef in libcamera::BoundMethodMember<libcamera::VimcCameraData, void, libcamera::FrameBuffer*>::activate(libcamera::FrameBuffer*, bool) ../include/libcamera/base/bound_method.h:194 #3 0x7f22515cc91f in libcamera::Signal<libcamera::FrameBuffer*>::emit(libcamera::FrameBuffer*) ../include/libcamera/base/signal.h:126 #4 0x7f22515c3305 in libcamera::V4L2VideoDevice::streamOff() ../src/libcamera/v4l2_videodevice.cpp:1605 #5 0x7f225181f345 in libcamera::PipelineHandlerVimc::stop(libcamera::Camera*) ../src/libcamera/pipeline/vimc/vimc.cpp:365 The VimcCameraData::bufferReady seems to emit even after the stream is stopped. It's primarily due to vimc's lack of handling FrameMetadata::FrameCancelled in its pipeline handler. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-16ipa: vimc: Send and retrieve FrameBuffers from IPAUmang Jain
Plumb through VIMC mojo interface to enable buffers passing. VIMC does not have parameters or statistics buffers but we can mimick the typical case of passing IPA buffers from pipeline handler to IPA using mock buffers. The mock IPA buffers are FrameBuffers which are dmabuf backed (in other words, mmap()able through MappedFramebuffer inside the IPA). This commits shows: - Passing the parameter buffer from the pipeline handler to the IPA through functions defined in mojom interface. - Passing request controls ControlList to the IPA. Any tests using VIMC will now loop in the IPA paths. Any tests running in isolated mode will help us to test IPA IPC code paths especially around (de)serialization of data passing from pipeline handlers to the IPA. Future IPA interface tests can simply extend the vimc mojom interface to achieve/test a specific use case as required. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-16ipa: vimc: Map and unmap buffersUmang Jain
VIMC pipeline handler has dmabuf-backed mock FrameBuffers which are specifically targetted mimicking IPA buffers (parameter and statistics). Map these mock buffers to the VIMC IPA that would enable exercising IPA IPC code paths. This will provide leverage to our test suite to test IPA IPC code paths, which are common to various platforms. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-16pipeline: vimc: Allocate mock IPA buffersUmang Jain
VIMC is a virtual test driver that doesn't have statistics or parameters buffers that are typically passed from a pipeline handler to its platform IPA. To increase the test coverage going forward, we can at least mimick the typical interaction of how a pipeline handler and IPA interact, and use it to increase the test coverage. Hence, create simple (single plane) dmabuf-backed FrameBuffers, which can act as mock IPA buffers and can be memory mapped (mmap) to VIMC IPA. To create these buffers, temporarily hijack the output video node and configure it with a V4L2DeviceFormat. Buffers then can be exported from the output video node using V4L2VideoDevice::exportBuffers(). These buffers will be mimicked as IPA buffers in subsequent commits. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-08-16ipa: vimc: Add configure() functionLaurent Pinchart
As part of an effort to make the vimc IPA usable for testing, extend it with a configure function. The configuration is currently ignored by the IPA. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2021-08-12libcamera: ipu3: Initialize controls in the IPAJacopo Mondi
All the IPU3 Camera controls are currently initialized by the pipeline handler which initializes them using the camera sensor configuration and platform specific requirements. However, some controls are better initialized by the IPA, which might, in example, cap the exposure times and frame duration to the constraints of its algorithms implementation. Also, moving forward, the IPA should register controls to report its capabilities, in example the ability to enable/disable 3A algorithms on request. Move the existing controls initialization to the IPA, by providing the sensor configuration and its controls to the IPU3IPA::init() function, which initializes controls and returns them to the pipeline through an output parameter. The existing controls initialization has been copied verbatim from the pipeline handler to the IPA, if not a for few line breaks adjustments and the resulting Camera controls values are not changed. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-08-12libcamera: controls: Create ControlInfoMap with ControlIdMapJacopo Mondi
ControlInfoMap does not have a ControlId map associated, but rather creates one with the generateIdMap() function at creation time. As a consequence, when in the need to de-serialize a ControlInfoMap all the ControlId it contains are created by the deserializer instance, not being able to discern if the controls the ControlIdMap refers to are the global libcamera controls (and properties) or instances local to the V4L2 device that has first initialized the controls. As a consequence the ControlId stored in a de-serialized map will always be newly created entities, preventing lookup by ControlId reference on a de-serialized ControlInfoMap. In order to make it possible to use globally available ControlId instances whenever possible, create ControlInfoMap with a reference to an externally allocated ControlIdMap instead of generating one internally. As a consequence the class constructors take and additional argument, which might be not pleasant to type in, but enforces the concepts that ControlInfoMap should be created with controls part of the same id map. As the ControlIdMap the ControlInfoMap refers to needs to be allocated externally: - Use the globally available controls::controls (or properties::properties) id map when referring to libcamera controls - The V4L2 device that creates ControlInfoMap by parsing the device's controls has to allocate a ControlIdMap - The ControlSerializer that de-serializes a ControlInfoMap has to create and store the ControlIdMap the de-serialized info map refers to Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-10src: Remove all unused sys/mman.h inclusionsKieran Bingham
Remove leftover inclusions of the sys/mman header file. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-09libcamera: Rename 'method' to 'function'Laurent Pinchart
Usage of 'method' to refer to member functions comes from Java. The C++ standard uses the term 'function' only. Replace 'method' with 'function' or 'member function' through the whole code base and documentation. While at it, fix two typos (s/backeng/backend/). The BoundMethod and Object::invokeMethod() are left as-is here, and will be addressed separately. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-09ipu3: Disallow raw only camera configurationUmang Jain
To capture raw frames, the ImgU isn't needed. However, to implement auto-exposure, we do need to configure the IPA since it shall setup the sensor controls (exposure, vblank and so on) for the capture. One cannot simply configure the IPA, without the ImgU as the parameters and statistics buffer passed to the IPA are actually managed by the ImgU. Until we prepare and setup the ImgU to run an internal queue for raw-only camera configuration, disallow this configuration and report it as invalid. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>