libcamera/jmondi/libcamera.git - Jacopo Mondi's clone of libcamera

Age	Commit message (Expand)	Author
2020-01-13	rkisp1: add pipeline test for rkisp1	Show Liu
2019-10-23	libcamera: Standardise on C compatibility headers	Laurent Pinchart
2019-08-19	libcamera: camera_manager: Construct CameraManager instances manually	Laurent Pinchart
2019-05-23	meson: Create and use a dependency for libcamera and its headers	Laurent Pinchart
2019-05-23	meson: Fix coding style in meson.build files	Laurent Pinchart
2019-05-17	libcamera: media_device: Open and close media device inside populate()	Niklas Söderlund
2019-01-24	libcamera: device_enumerator: Reference-count MediaDevice instances	Laurent Pinchart
2019-01-22	libcamera: Global s/devnode/deviceNode rename	Jacopo Mondi
2019-01-22	test: pipeline: IPU3: Add IPU3 pipeline test	Jacopo Mondi

/* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2018, Google Inc. * * camera.cpp - Camera device */ #include <libcamera/camera.h> #include <array> #include <atomic> #include <iomanip> #include <libcamera/base/log.h> #include <libcamera/base/thread.h> #include <libcamera/framebuffer_allocator.h> #include <libcamera/request.h> #include <libcamera/stream.h> #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_controls.h" #include "libcamera/internal/pipeline_handler.h" /** * \file libcamera/camera.h * \brief Camera device handling * * \page camera-model Camera Model * * libcamera acts as a middleware between applications and camera hardware. It * provides a solution to an unsolvable problem: reconciling applications, * which need to run on different systems without dealing with device-specific * details, and camera hardware, which exhibits a wide variety of features, * limitations and architecture variations. In order to do so, it creates an * abstract camera model that hides the camera hardware from applications. The * model is designed to strike the right balance between genericity, to please * generic applications, and flexibility, to expose even the most specific * hardware features to the most demanding applications. * * In libcamera, a Camera is defined as a device that can capture frames * continuously from a camera sensor and store them in memory. If supported by * the device and desired by the application, the camera may store each * captured frame in multiple copies, possibly in different formats and sizes. * Each of these memory outputs of the camera is called a Stream. * * A camera contains a single image source, and separate camera instances * relate to different image sources. For instance, a phone containing front * and back image sensors will be modelled with two cameras, one for each * sensor. When multiple streams can be produced from the same image source, * all those streams are guaranteed to be part of the same camera. * * While not sharing image sources, separate cameras can share other system * resources, such as ISPs. For this reason camera instances may not be fully * independent, in which case usage restrictions may apply. For instance, a * phone with a front and a back camera may not allow usage of the two cameras * simultaneously. * * The camera model defines an implicit pipeline, whose input is the camera * sensor, and whose outputs are the streams. Along the pipeline, the frames * produced by the camera sensor are transformed by the camera into a format * suitable for applications, with image processing that improves the quality * of the captured frames. The camera exposes a set of controls that * applications may use to manually control the processing steps. This * high-level camera model is the minimum baseline that all cameras must * conform to. * * \section camera-pipeline-model Pipeline Model * * Camera hardware differs in the supported image processing operations and the * order in which they are applied. The libcamera pipelines abstract the * hardware differences and expose a logical view of the processing operations * with a fixed order. This offers low-level control of those operations to * applications, while keeping application code generic. * * Starting from the camera sensor, a pipeline applies the following * operations, in that order. * * - Pixel exposure * - Analog to digital conversion and readout * - Black level subtraction * - Defective pixel correction * - Lens shading correction * - Spatial noise filtering * - Per-channel gains (white balance) * - Demosaicing (color filter array interpolation) * - Color correction matrix (typically RGB to RGB) * - Gamma correction * - Color space transformation (typically RGB to YUV) * - Cropping * - Scaling * * Not all cameras implement all operations, and they are not necessarily * implemented in the above order at the hardware level. The libcamera pipeline * handlers translate the pipeline model to the real hardware configuration. * * \subsection digital-zoom Digital Zoom * * Digital zoom is implemented as a combination of the cropping and scaling * stages of the pipeline. Cropping is controlled explicitly through the * controls::ScalerCrop control, while scaling is controlled implicitly based * on the crop rectangle and the output stream size. The crop rectangle is * expressed relatively to the full pixel array size and indicates how the field * of view is affected by the pipeline. */ namespace libcamera { LOG_DECLARE_CATEGORY(Camera) /** * \class CameraConfiguration * \brief Hold configuration for streams of the camera * The CameraConfiguration holds an ordered list of stream configurations. It * supports iterators and operates as a vector of StreamConfiguration instances. * The stream configurations are inserted by addConfiguration(), and the * at() function or operator[] return a reference to the StreamConfiguration * based on its insertion index. Accessing a stream configuration with an * invalid index results in undefined behaviour. * * CameraConfiguration instances are retrieved from the camera with * Camera::generateConfiguration(). Applications may then inspect the * configuration, modify it, and possibly add new stream configuration entries * with addConfiguration(). Once the camera configuration satisfies the * application, it shall be validated by a call to validate(). The validation * implements "try" semantics: it adjusts invalid configurations to the closest * achievable parameters instead of rejecting them completely. Applications * then decide whether to accept the modified configuration, or try again with * a different set of parameters. Once the configuration is valid, it is passed * to Camera::configure(). */ /** * \enum CameraConfiguration::Status * \brief Validity of a camera configuration * \var CameraConfiguration::Valid * The configuration is fully valid * \var CameraConfiguration::Adjusted * The configuration has been adjusted to a valid configuration * \var CameraConfiguration::Invalid * The configuration is invalid and can't be adjusted automatically */ /** * \typedef CameraConfiguration::iterator * \brief Iterator for the stream configurations in the camera configuration */ /** * \typedef CameraConfiguration::const_iterator * \brief Const iterator for the stream configuration in the camera * configuration */ /** * \brief Create an empty camera configuration */ CameraConfiguration::CameraConfiguration() : transform(Transform::Identity), config_({}) { } CameraConfiguration::~CameraConfiguration() { } /** * \brief Add a stream configuration to the camera configuration * \param[in] cfg The stream configuration */ void CameraConfiguration::addConfiguration(const StreamConfiguration &cfg) { config_.push_back(cfg); } /** * \fn CameraConfiguration::validate() * \brief Validate and possibly adjust the camera configuration * * This function adjusts the camera configuration to the closest valid * configuration and returns the validation status. * * \todo: Define exactly when to return each status code. Should stream * parameters set to 0 by the caller be adjusted without returning Adjusted ? * This would potentially be useful for applications but would get in the way * in Camera::configure(). Do we need an extra status code to signal this ? * * \todo: Handle validation of buffers count when refactoring the buffers API. * * \return A CameraConfiguration::Status value that describes the validation * status. * \retval CameraConfiguration::Invalid The configuration is invalid and can't * be adjusted. This may only occur in extreme cases such as when the * configuration is empty. * \retval CameraConfigutation::Adjusted The configuration has been adjusted * and is now valid. Parameters may have changed for any stream, and stream * configurations may have been removed. The caller shall check the * configuration carefully. * \retval CameraConfiguration::Valid The configuration was already valid and * hasn't been adjusted. */ /** * \brief Retrieve a reference to a stream configuration * \param[in] index Numerical index * * The \a index represents the zero based insertion order of stream * configuration into the camera configuration with addConfiguration(). Calling * this function with an invalid index results in undefined behaviour. * * \return The stream configuration */ StreamConfiguration &CameraConfiguration::at(unsigned int index) { return config_[index]; } /** * \brief Retrieve a const reference to a stream configuration * \param[in] index Numerical index * * The \a index represents the zero based insertion order of stream * configuration into the camera configuration with addConfiguration(). Calling * this function with an invalid index results in undefined behaviour. * * \return The stream configuration */ const StreamConfiguration &CameraConfiguration::at(unsigned int index) const { return config_[index]; } /** * \fn StreamConfiguration &CameraConfiguration::operator[](unsigned int) * \brief Retrieve a reference to a stream configuration * \param[in] index Numerical index * * The \a index represents the zero based insertion order of stream * configuration into the camera configuration with addConfiguration(). Calling * this function with an invalid index results in undefined behaviour. * * \return The stream configuration */ /** * \fn const StreamConfiguration &CameraConfiguration::operator[](unsigned int) const * \brief Retrieve a const reference to a stream configuration * \param[in] index Numerical index * * The \a index represents the zero based insertion order of stream * configuration into the camera configuration with addConfiguration(). Calling * this function with an invalid index results in undefined behaviour. * * \return The stream configuration */ /** * \brief Retrieve an iterator to the first stream configuration in the * sequence * \return An iterator to the first stream configuration */ CameraConfiguration::iterator CameraConfiguration::begin() { return config_.begin(); } /** * \brief Retrieve a const iterator to the first element of the stream * configurations * \return A const iterator to the first stream configuration */ CameraConfiguration::const_iterator CameraConfiguration::begin() const { return config_.begin(); } /** * \brief Retrieve an iterator pointing to the past-the-end stream * configuration in the sequence * \return An iterator to the element following the last stream configuration */ CameraConfiguration::iterator CameraConfiguration::end() { return config_.end(); } /** * \brief Retrieve a const iterator pointing to the past-the-end stream * configuration in the sequence * \return A const iterator to the element following the last stream * configuration */ CameraConfiguration::const_iterator CameraConfiguration::end() const { return config_.end(); } /** * \brief Check if the camera configuration is empty * \return True if the configuration is empty */ bool CameraConfiguration::empty() const { return config_.empty(); } /** * \brief Retrieve the number of stream configurations * \return Number of stream configurations */ std::size_t CameraConfiguration::size() const { return config_.size(); } /** * \var CameraConfiguration::transform * \brief User-specified transform to be applied to the image * * The transform is a user-specified 2D plane transform that will be applied * to the camera images by the processing pipeline before being handed to * the application. This is subsequent to any transform that is already * required to fix up any platform-defined rotation. * * The usual 2D plane transforms are allowed here (horizontal/vertical * flips, multiple of 90-degree rotations etc.), but the validate() function * may adjust this field at its discretion if the selection is not supported. */ /** * \var CameraConfiguration::config_ * \brief The vector of stream configurations */ /** * \class Camera::Private * \brief Base class for camera private data * * The Camera::Private class stores all private data associated with a camera. * In addition to hiding core Camera data from the public API, it is expected to * be subclassed by pipeline handlers to store pipeline-specific data. * * Pipeline handlers can obtain the Camera::Private instance associated with a * camera by calling Camera::_d(). */ /** * \brief Construct a Camera::Private instance * \param[in] pipe The pipeline handler responsible for the camera device */ Camera::Private::Private(PipelineHandler *pipe) : requestSequence_(0), pipe_(pipe->shared_from_this()), disconnected_(false), state_(CameraAvailable) { } Camera::Private::~Private() { if (state_.load(std::memory_order_acquire) != Private::CameraAvailable) LOG(Camera, Error) << "Removing camera while still in use"; } /** * \fn Camera::Private::pipe() * \brief Retrieve the pipeline handler related to this camera * \return The pipeline handler that created this camera */ /** * \fn Camera::Private::validator() * \brief Retrieve the control validator related to this camera * \return The control validator associated with this camera */ /** * \var Camera::Private::queuedRequests_ * \brief The list of queued and not yet completed requests * * This list tracks requests queued in order to ensure completion of all * requests when the pipeline handler is stopped. * * \sa PipelineHandler::queueRequest(), PipelineHandler::stop(), * PipelineHandler::completeRequest() */ /** * \var Camera::Private::controlInfo_ * \brief The set of controls supported by the camera * * The control information shall be initialised by the pipeline handler when * creating the camera. * * \todo This member was initially meant to stay constant after the camera is * created. Several pipeline handlers are already updating it when the camera * is configured. Update the documentation accordingly, and possibly the API as * well, when implementing official support for control info updates. */ /** * \var Camera::Private::properties_ * \brief The list of properties supported by the camera * * The list of camera properties shall be initialised by the pipeline handler * when creating the camera, and shall not be modified afterwards. */ /** * \var Camera::Private::requestSequence_ * \brief The queuing sequence number of the request * * When requests are queued, they are given a per-camera sequence number to * facilitate debugging of internal request usage. * * The requestSequence_ tracks the number of requests queued to a camera * over its lifetime. */ static const char *const camera_state_names[] = { "Available", "Acquired", "Configured", "Stopping", "Running", }; bool Camera::Private::isRunning() const { return state_.load(std::memory_order_acquire) == CameraRunning; } int Camera::Private::isAccessAllowed(State state, bool allowDisconnected, const char *from) const { if (!allowDisconnected && disconnected_) return -ENODEV; State currentState = state_.load(std::memory_order_acquire); if (currentState == state) return 0; ASSERT(static_cast<unsigned int>(state) < std::size(camera_state_names)); LOG(Camera, Error) << "Camera in " << camera_state_names[currentState] << " state trying " << from << "() requiring state " << camera_state_names[state]; return -EACCES; } int Camera::Private::isAccessAllowed(State low, State high, bool allowDisconnected, const char *from) const { if (!allowDisconnected && disconnected_) return -ENODEV; State currentState = state_.load(std::memory_order_acquire); if (currentState >= low && currentState <= high) return 0; ASSERT(static_cast<unsigned int>(low) < std::size(camera_state_names) && static_cast<unsigned int>(high) < std::size(camera_state_names)); LOG(Camera, Error) << "Camera in " << camera_state_names[currentState] << " state trying " << from << "() requiring state between " << camera_state_names[low] << " and " << camera_state_names[high]; return -EACCES; } void Camera::Private::disconnect() { /* * If the camera was running when the hardware was removed force the * state to Configured state to allow applications to free resources * and call release() before deleting the camera. */ if (state_.load(std::memory_order_acquire) == Private::CameraRunning) state_.store(Private::CameraConfigured, std::memory_order_release); disconnected_ = true; } void Camera::Private::setState(State state) { state_.store(state, std::memory_order_release); } /** * \class Camera * \brief Camera device * * \todo Add documentation for camera start timings. What exactly does the * camera expect the pipeline handler to do when start() is called? * * The Camera class models a camera capable of producing one or more image