libcamera/vivid.git - libcamera pipeline handler for VIVID

diff options

Diffstat (limited to 'src/gstreamer/gstlibcamerasrc.cpp')

0 files changed, 0 insertions, 0 deletions

.. SPDX-License-Identifier: CC-BY-SA-4.0 IPA Writer's Guide ================== IPA modules are Image Processing Algorithm modules. They provide functionality that the pipeline handler can use for image processing. This guide covers the definition of the IPA interface, and how to plumb the connection between the pipeline handler and the IPA. The IPA interface and protocol ------------------------------ The IPA interface defines the interface between the pipeline handler and the IPA. Specifically, it defines the functions that the IPA exposes that the pipeline handler can call, and the signals that the pipeline handler can connect to, in order to receive data from the IPA asynchronously. In addition, it contains any custom data structures that the pipeline handler and IPA may pass to each other. It is possible to use the same IPA interface with multiple pipeline handlers on different hardware platforms. Generally in such cases, these platforms would have a common hardware ISP pipeline. For instance, the rkisp1 pipeline handler supports both the RK3399 and the i.MX8MP as they integrate the same ISP. However, the i.MX8MP has a more complex camera pipeline, which may call for a dedicated pipeline handler in the future. As the ISP is the same as for RK3399, the same IPA interface could be used for both pipeline handlers. The build files provide a mapping from pipeline handler to the IPA interface name as detailed in :ref:`compiling-section`. The IPA protocol refers to the agreement between the pipeline handler and the IPA regarding the expected response(s) from the IPA for given calls to the IPA. This protocol doesn't need to be declared anywhere in code, but it shall be documented, as there may be multiple IPA implementations for one pipeline handler. As part of the design of libcamera, IPAs may be isolated in a separate process, or run in the same process but a different thread from libcamera. The pipeline handler and IPA shall not have to change their operation based on whether the IPA is isolated or not, but the possibility of isolation needs to be kept in mind. Therefore all data that is passed between them must be serializable, so they must be defined separately in the `mojo Interface Definition Language`_ (IDL), and a code generator will generate headers and serializers corresponding to the definitions. Every interface is defined in a mojom file and includes: - the functions that the pipeline handler can call from the IPA - signals in the pipeline handler that the IPA can emit - any data structures that are to be passed between the pipeline handler and the IPA All IPA modules of a given pipeline handler use the same IPA interface. The IPA interface definition is thus written by the pipeline handler author, based on how they design the interactions between the pipeline handler and the IPA. The entire IPA interface, including the functions, signals, and any custom structs shall be defined in a file named {interface_name}.mojom under include/libcamera/ipa/. .. _mojo Interface Definition Language: https://chromium.googlesource.com/chromium/src.git/+/master/mojo/public/tools/bindings/README.md Namespacing ----------- To avoid name collisions between data types defined by different IPA interfaces and data types defined by libcamera, each IPA interface must be defined in its own namespace. The namespace is specific with mojo's module directive. It must be the first non-comment line in the mojo data definition file. For example, the Raspberry Pi IPA interface uses: .. code-block:: none module ipa.rpi; This will become the ipa::rpi namespace in C++ code. Data containers --------------- Since the data passed between the pipeline handler and the IPA must support serialization, any custom data containers must be defined with the mojo IDL. The following list of libcamera objects are supported in the interface definition, and may be used as function parameter types or struct field types: - libcamera.ControlInfoMap - libcamera.ControlList - libcamera.FileDescriptor - libcamera.IPABuffer - libcamera.IPACameraSensorInfo - libcamera.IPASettings - libcamera.IPAStream - libcamera.Point - libcamera.Rectangle - libcamera.Size - libcamera.SizeRange To use them, core.mojom must be included in the mojo data definition file: .. code-block:: none import "include/libcamera/ipa/core.mojom"; Other custom structs may be defined and used as well. There is no requirement that they must be defined before usage. enums and structs are supported. The following is an example of a definition of an enum, for the purpose of being used as flags: .. code-block:: none enum ConfigParameters { ConfigLsTable = 0x01, ConfigStaggeredWrite = 0x02, ConfigSensor = 0x04, ConfigDropFrames = 0x08, }; The following is an example of a definition of a struct: .. code-block:: none struct ConfigInput { uint32 op; uint32 transform; libcamera.FileDescriptor lsTableHandle; int32 lsTableHandleStatic = -1; map<uint32, libcamera.IPAStream> streamConfig; array<libcamera.IPABuffer> buffers; }; This example has some special things about it. First of all, it uses the FileDescriptor data type. This type must be used to ensure that the file descriptor that it contains is translated properly across the IPC boundary (when the IPA is in an isolated process). This does mean that if the file descriptor should be sent without being translated (for example, for the IPA to tell the pipeline handler which fd *that the pipeline handler holds* to act on), then it must be in a regular int32 type. This example also illustrates that struct fields may have default values, as is assigned to lsTableHandleStatic. This is the value that the field will take when the struct is constructed with the default constructor. Arrays and maps are supported as well. They are translated to C++ vectors and maps, respectively. The members of the arrays and maps are embedded, and cannot be const. Note that nullable fields, static-length arrays, handles, and unions, which are supported by mojo, are not supported by our code generator. The Main IPA interface ---------------------- The IPA interface is split in two parts, the Main IPA interface, which describes the functions that the pipeline handler can call from the IPA, and the Event IPA interface, which describes the signals received by the pipeline handler that the IPA can emit. Both must be defined. This section focuses on the Main IPA interface. The main interface must be named as IPA{interface_name}Interface. The functions that the pipeline handler can call from the IPA may be synchronous or asynchronous. Synchronous functions do not return until the IPA returns from the function, while asynchronous functions return immediately without waiting for the IPA to return. At a minimum, the following three functions must be present (and implemented): - init(); - start(); - stop(); All three of these functions are synchronous. The parameters for start() and init() may be customized. init() initializes the IPA interface. It shall be called before any other function of the IPAInterface. stop() informs the IPA module that the camera is stopped. The IPA module shall release resources prepared in start(). A configure() function is recommended. Any ControlInfoMap instances that will be used by the IPA must be sent to the IPA from the pipeline handler, at configure time, for example. All input parameters will become const references, except for arithmetic types, which will be passed by value. Output parameters will become pointers, unless the first output parameter is an int32, or there is only one primitive output parameter, in which case it will become a regular return value. const is not allowed inside of arrays and maps. mojo arrays will become C++ std::vector<>. By default, all functions defined in the main interface are synchronous. This means that in the case of IPC (i.e. isolated IPA), the function call will not return until the return value or output parameters are ready. To specify an asynchronous function, the [async] attribute can be used. Asynchronous functions must not have any return value or output parameters, since in the case of IPC the call needs to return immediately. It is also possible that the IPA will not be run in isolation. In this case, the IPA thread will not exist until start() is called. This means that in the case of no isolation, asynchronous calls cannot be made before start(). Since the IPA interface must be the same regardless of isolation, the same restriction applies to the case of isolation, and any function that will be called before start() must be synchronous. In addition, any call made after start() and before stop() must be asynchronous. The motivation for this is to avoid damaging real-time


context:
space:
mode: