1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
|
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/*
* Copyright (C) 2022, Google Inc.
*
* IPA Frame context queue
*/
#include "fc_queue.h"
#include <libcamera/base/log.h>
namespace libcamera {
LOG_DEFINE_CATEGORY(FCQueue)
namespace ipa {
/**
* \file fc_queue.h
* \brief Queue of per-frame contexts
*/
/**
* \struct FrameContext
* \brief Context for a frame
*
* The frame context stores data specific to a single frame processed by the
* IPA module. Each frame processed by the IPA module has a context associated
* with it, accessible through the Frame Context Queue.
*
* Fields in the frame context should reflect values and controls associated
* with the specific frame as requested by the application, and as configured by
* the hardware. Fields can be read by algorithms to determine if they should
* update any specific action for this frame, and finally to update the metadata
* control lists when the frame is fully completed.
*
* \var FrameContext::frame
* \brief The frame number
*/
/**
* \fn FrameContext::init()
* \brief Initialize a frame context
* \param[in] frameNum The frame number to assign to this FrameContext
*
* This function initializes a frame context by assigning it a frame number.
* The single IPA modules are expected to override this function to initialize
* their derived FrameContext implementation to their desired default values.
*/
/**
* \class FCQueue
* \brief A support class for managing FrameContext instances in IPA modules
* \tparam FrameContext The IPA module-specific FrameContext derived class type
*
* Along with the Module and Algorithm classes, the frame context queue is a
* core component of the libipa infrastructure. It stores per-frame contexts
* used by the Algorithm operations. By centralizing the lifetime management of
* the contexts and implementing safeguards against underflows and overflows, it
* simplifies IPA modules and improves their reliability.
*
* The queue references frame contexts by a monotonically increasing sequence
* number. The FCQueue design assumes that this number matches both the sequence
* number of the corresponding frame, as generated by the camera sensor, and the
* sequence number of the request. This allows IPA modules to obtain the frame
* context from any location where a request or a frame is available.
*
* A frame context normally begins its lifetime when the corresponding request
* is queued, way before the frame is captured by the camera sensor. IPA modules
* allocate the context from the queue at that point, calling alloc() using the
* request number. The queue initializes the context, and the IPA module then
* populates it with data from the request. The context can be later retrieved
* with a call to get(), typically when the IPA module is requested to provide
* sensor or ISP parameters or receives statistics for a frame. The frame number
* is used at that point to identify the context.
*
* If an application fails to queue requests to the camera fast enough, frames
* may be produced by the camera sensor and processed by the IPA module without
* a corresponding request having been queued to the IPA module. This creates an
* underrun condition, where the IPA module will try to get a frame context that
* hasn't been allocated. In this case, the get() function will allocate and
* initialize a context for the frame, and log a message. Algorithms will not
* apply the controls associated with the late request, but should otherwise
* behave correctly.
*
* \todo Mark the frame context with a per-frame control error flag in case of
* underrun, and research how algorithms should handle this.
*
* At its core, the queue uses a circular buffer to avoid dynamic memory
* allocation at runtime. The buffer is pre-allocated with a maximum number of
* entries when the FCQueue instance is constructed. Entries are initialized on
* first use by alloc() or, in underrun conditions, get(). The queue is not
* allowed to overflow, which must be ensured by pipeline handlers never
* queuing more in-flight requests to the IPA module than the queue size. If an
* overflow condition is detected, the queue will log a fatal error.
*
* IPA module-specific frame context implementations shall inherit from the
* FrameContext base class to support the minimum required features for a
* FrameContext.
*/
/**
* \fn FCQueue::FCQueue(unsigned int size)
* \brief Construct a frame contexts queue of a specified size
* \param[in] size The number of contexts in the queue
*/
/**
* \fn FCQueue::clear()
* \brief Clear the contexts queue
*
* IPA modules must clear the frame context queue at the beginning of a new
* streaming session, in IPAModule::start().
*
* \todo Fix any issue this may cause with requests queued before the camera is
* started.
*/
/**
* \fn FCQueue::alloc(uint32_t frame)
* \brief Allocate and return a FrameContext for the \a frame
* \param[in] frame The frame context sequence number
*
* The first call to obtain a FrameContext from the FCQueue should be handled
* through this function. The FrameContext will be initialised, if not
* initialised already, and returned to the caller.
*
* If the FrameContext was already initialized for this \a frame, a warning will
* be reported and the previously initialized FrameContext is returned.
*
* Frame contexts are expected to be initialised when a Request is first passed
* to the IPA module in IPAModule::queueRequest().
*
* \return A reference to the FrameContext for sequence \a frame
*/
/**
* \fn FCQueue::get(uint32_t frame)
* \brief Obtain the FrameContext for the \a frame
* \param[in] frame The frame context sequence number
*
* If the FrameContext is not correctly initialised for the \a frame, it will be
* initialised.
*
* \return A reference to the FrameContext for sequence \a frame
*/
} /* namespace ipa */
} /* namespace libcamera */
|
