summaryrefslogtreecommitdiff
path: root/test/serialization/generated_serializer
diff options
context:
space:
mode:
authorTomi Valkeinen <tomi.valkeinen@ideasonboard.com>2023-03-13 09:28:13 +0200
committerKieran Bingham <kieran.bingham@ideasonboard.com>2023-03-21 23:33:43 +0000
commitb3cefd4c0e92d96c730410cff5ddb69b28d693fd (patch)
treeb4b5a2284f12946f0d0f801b7bfb12f0e5bba71d /test/serialization/generated_serializer
parentafc5ea57b496c633a1a16c67cf132df6c5ed9b46 (diff)
py: cam: Fix demosaic overflow issue
The demosaic code first expands the buffer datatype to uint16, and then shifts the data left so that the 8, 10 and 12 bitspp formats all become 16 bitspp. It then, eventually, uses np.einsum to calculate averages, but this averaging sums multiple uint16 values together, and stores them in uint16 storage. As in the first step we shifted the values left, possibly getting values close to the maximum of uint16 range, we, of course, overflow when summing them together. This leads to rather bad looking images. Fix this by dropping the original shift. It serves no purpose, and is probably a remnant of some early testing code. This way the largest numbers we are summing together are 12 bit values, and as we use a 3x3 window from which we fetch values, for a single rgb plane, the max number of 12 bit values is 5 (for green). Sum of 5 12 bit values is well below the 16 bit maximum. Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Diffstat (limited to 'test/serialization/generated_serializer')
0 files changed, 0 insertions, 0 deletions