summaryrefslogtreecommitdiff
path: root/src/android/jpeg
AgeCommit message (Collapse)Author
2021-09-21android: jpeg: Rectify variable naming style.Umang Jain
Rectify variable renaming style for YPlaneSize, UVPlaneSize. libcamera uses camelCase where first letter should be in lower case. Fixes: e355ca0087cd9("android: jpeg: Split and pass the thumbnail planes to encoder") Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-09-09android: jpeg: Split and pass the thumbnail planes to encoderUmang Jain
After multi-planar support was introduced for jpeg encoding as well, EncoderLibJpeg::encode() expects a vector of planes as the source of framebuffer to be encoded. Currently, we are passing a contiguous buffer which is treated as only one plane (instead of two, as thumbnail is NV12). Hence, split the thumbnail data into respective planes according to NV12. This fixes a crash in encoding of thumbnails. Fixes: 894ca69f6043("android: jpeg: Support multi-planar buffers") Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-09-07android: jpeg: Support multi-planar buffersLaurent Pinchart
The JPEG post-processor uses MappedFrameBuffer to access pixel data, but only uses data from the first plane. Pass the vector of planes to the encode() function to correctly handle multi-planar formats (currently limited to NV12). Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Acked-by: Umang Jain <umang.jain@ideasonboard.com> Tested-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-09-07android: jpeg: Use stride instead of image width for line addressLaurent Pinchart
When calculating the luma line address, the image width is used instead of the stride. Without padding at the end of the line the the values should be identical, but this is conceptually incorrect in any case. Fix it. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-08-31libcamera: mapped_framebuffer: Rename maps() to planes()Hirokazu Honda
MappedFrameBuffer::maps() returns planes_. This renames the function name to planes(). Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-10libcamera: MappedFrameBuffer: Use typed Flags<MapModes>Kieran Bingham
Remove the need for callers to reference PROT_READ/PROT_WRITE directly from <sys/mman.h> by instead exposing the Read/Write mapping options as flags from the MappedFrameBuffer class itself. While here, introduce the <stdint.h> header which is required for the uint8_t as part of the Plane. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-10libcamera: Give MappedFrameBuffer its own implementationKieran Bingham
The MappedFrameBuffer is a convenience feature which sits on top of the FrameBuffer and facilitates mapping it to CPU accessible memory with mmap. This implementation is internal and currently sits in the same internal files as the internal FrameBuffer, thus exposing those internals to users of the MappedFramebuffer implementation. Move the MappedFrameBuffer and MappedBuffer implementation to its own implementation files, and fix the sources throughout to use that accordingly. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-09libcamera: Rename 'method' to 'function'Laurent Pinchart
Usage of 'method' to refer to member functions comes from Java. The C++ standard uses the term 'function' only. Replace 'method' with 'function' or 'member function' through the whole code base and documentation. While at it, fix two typos (s/backeng/backend/). The BoundMethod and Object::invokeMethod() are left as-is here, and will be addressed separately. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-02android: jpeg: get ISO from SENSOR_SENSITIVITYPaul Elder
The data for the exif ISO tag needs to come from SENSOR_SENSITIVITY. Set it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-07-11libcamera: buffer: Rename buffer.h to framebuffer.hLaurent Pinchart
libcamera names header files based on the classes they define. The buffer.h file is an exception. Rename it to framebuffer.h. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Move span to base libraryKieran Bingham
Move span, and adjust the Doxygen exclusion as well. Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Move extended base functionalityKieran Bingham
Move the functionality for the following components to the new base support library: - BoundMethod - EventDispatcher - EventDispatcherPoll - Log - Message - Object - Signal - Semaphore - Thread - Timer While it would be preferable to see these split to move one component per commit, these components are all interdependent upon each other, which leaves us with one big change performing the move for all of them. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-06-25libcamera/base: Move utils to the base libraryKieran Bingham
Move the utils functionality to the libcamera/base library. Reviewed-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-05-14android: jpeg: post_processor: Use the new metadata functionsPaul Elder
Now that CameraMetadata supports more convenient functions, use those instead. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-03-10android: jpeg: exif: change GPS method encoding from ASCII to NoEncodingPaul Elder
According to the EXIF specification, the GPS method should be UNDEFINED, and the first 8 bytes will designate the type. However, CTS expects the first 8 bytes to be part of the data. Remove the 8-byte encoding designator by changing the encoding to NoEncoding to appease CTS. This is part of the fix that allows the following CTS test to pass: - android.hardware.cts.CameraTest#testJpegExif Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-03-10android: jpeg: exif: Fix setGPSLocation longitudePaul Elder
There was a copy-paste error that caused the latitude to be set twice and the longitude never. Fix this. This is part of the fix that allows the following CTS test to pass: - android.hardware.cts.CameraTest#testJpegExif Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-03-10android: jpeg: exif: Simplify setGPSDateTimestamp and setGPSDMSPaul Elder
Now that setRational() supports setting multiple rational values, use that in setGPSDateTimestamp and setGPSDMS which previously set every rational manually. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-03-10android: jpeg: exif: Fix and expand setRationalPaul Elder
setRational was not working properly for EXIF tags in the GPS IFD due to libexif not supporting those tags in exif_entry_initialize(). Manually specify the size of the EXIF entry to fix this. While at it, add support for setting multiple rationals, as that is a common use case for rational EXIF tags. As Rational types are no longer initialized by libexif directly, the EXIF_TAG_{X,Y}_RESOLUTION exif tags will not have their default values populated. This allows the GPS altitude to be set properly, and is part of the fix to allow the following CTS test to pass: - android.hardware.cts.CameraTest#testJpegExif Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-03-03android: jpeg: Use CameraBuffer::jpegBufferSize()Jacopo Mondi
Use the newly introduced function to retrieve the size of the JPEG encoding destination buffer, in order to calculate where the JPEG_BLOB_ID should be placed. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
2021-03-03android: post_processor: Use CameraBuffer APIJacopo Mondi
Use the newly introduced CameraBuffer class as the type for the destination buffer in the PostProcessor class hierarchy in place of the libcamera::MappedFrameBuffer one and use its API to retrieve the length and the location of the CameraBuffer plane allocated for JPEG post-processing. Remove all the assumption on the underlying memory storage and only go through the CameraBuffer API when dealing with memory buffers. To do so rework the Encoder interface to use a raw pointer and an explicit size to remove access to the Span<uint8_t> maps that serve as memory storage for the current implementation but might not be ideal for other memory backend. Now that the whole PostProcessor hierarchy has been converted to use the CameraBuffer API remove libcamera::MappedBuffer as base class of the CameraBuffer interface and only reply on its interface. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
2021-02-02android: post_processor: Change the type destination in process()Hirokazu Honda
The type of the destination buffer in PostProcessor::process() is libcamera::Span. libcamera::Span is used for one dimension buffer (e.g. blob buffer). The destination can be multiple dimensions buffer (e.g. yuv frame). Therefore, this changes the type of the destination buffer to MappedFrameBuffer. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
2021-02-01android: camera_device: Fix exposure time tag in exif and androidPaul Elder
The ExposureTime libcamera control is in microseconds while android and our exif component use nanoseconds. Convert it appropriately. CTS also expects the ExposureTime exif tag to match the ExposureTime set in the android result metadata. Fix it. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2021-01-27android: jpeg: Set thumbnail and JPEG quality based on requestPaul Elder
Set the thumbnail quality and the JPEG quality based on the android request metadata. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-01-27android: jpeg: Configure thumbnailer based on request metadataPaul Elder
Configure the thumbnailer based on the thumbnail parameters given by the android request metadata. Only the thumbnail encoder needs to be configured, and since it is only used at post-processing time, move the configuration out of the post-processor constructor and into the processing step. Also set the following android result metadata tags: - ANDROID_JPEG_THUMBNAIL_SIZE - ANDROID_JPEG_THUMBNAIL_QUALITY Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-01-27android: Set result metadata and EXIF fields based on request metadataPaul Elder
Set the following android result metadata: - ANDROID_LENS_FOCAL_LENGTH - ANDROID_LENS_APERTURE - ANDROID_JPEG_GPS_TIMESTAMP - ANDROID_JPEG_GPS_COORDINATES - ANDROID_JPEG_GPS_PROCESSING_METHOD And the following EXIF fields: - GPSDatestamp - GPSTimestamp - GPSLocation - GPSLatitudeRef - GPSLatitude - GPSLongitudeRef - GPSLongitude - GPSAltitudeRef - GPSAltitude - GPSProcessingMethod - FocalLength - ExposureTime - FNumber - ISO - Flash - WhiteBalance - SubsecTime - SubsecTimeOriginal - SubsecTimeDigitized Based on android request metadata. This allows the following CTS tests to pass: - android.hardware.camera2.cts.StillCaptureTest#testFocalLengths - android.hardware.camera2.cts.StillCaptureTest#testJpegExif Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-01-27android: jpeg: exif: Add functions for setting various valuesPaul Elder
Add functions for setting the following EXIF fields: - GPSDatestamp - GPSTimestamp - GPSLocation - GPSLatitudeRef - GPSLatitude - GPSLongitudeRef - GPSLongitude - GPSAltitudeRef - GPSAltitude - GPSProcessingMethod - FocalLength - ExposureTime - FNumber - ISO - Flash - WhiteBalance - SubsecTime - SubsecTimeOriginal - SubsecTimeDigitized These are in preparation for fixing the following CTS tests: - android.hardware.camera2.cts.StillCaptureTest#testFocalLengths - android.hardware.camera2.cts.StillCaptureTest#testJpegExif Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-01-27android: jpeg: exif: Fix setOrientation EXIF valuesPaul Elder
The input to setOrientation is angle clockwise from the sensor orientation, while the EXIF output values were swapped for 90 and 270 degrees. From the EXIF spec: 6 = The 0th row is the visual right-hand side of the image, and the 0th column is the visual top. 8 = The 0th row is the visual left-hand side of the image, and the 0th column is the visual bottom. 6 should be 90 degrees clockwise, while 8 should 270 degrees clockwise. Fix this. As Android defines the rotation as the clockwise angle by which the image needs to be rotated to appear in the correct orientation on the device screen, the previous values would be correct if the input angle was from the camera orientation. Since the correct input should be the requested JPEG orientation, these new values are the correct ones. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-01-27android: jpeg: exif: Expand setString to support different encodingsPaul Elder
GPSProcessingMethod and UserComment in EXIF tags can be in UTF-16. Expand setString to take an encoding when the field type is undefined. Update callers accordingly. Signed-off-by: Paul Elder <paul.elder@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2020-11-17android: jpeg: Use LGPL-2.1 license for post_processor_jpeg.cppLaurent Pinchart
The JPEG post-processor is marked as licensed under GPL-2.0-or-later. This is an oversight and unvoluntary. License it under the LGPL-2.1-or-later as the rest of the camera HAL implementation. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Acked-by: Umang Jain <email@uajain.com> Acked-by: Hirokazu Honda <hiroh@chromium.org>
2020-10-28android: jpeg: post_processor_jpeg: Embed thumbnail into Exif metadataUmang Jain
Embed a Jpeg-encoded thumbnail into Exif metadata using the Thumbnailer class that got introduced. Introduce a helper function in Exif class for setting the thumbnail data. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> [Kieran: Add todo comment, and Compression enum] Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-28android: jpeg: Introduce a simple image thumbnailerUmang Jain
Add a basic image Thumbnailer class for the frames being captured. Currently, the thumbnailer can scale NV12 frames. It shall be used to generate a thumbnail image for EXIF metadata, in the subsequent commit. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-28android: jpeg: encoder_libjpeg: Allow encoding raw frame bytesUmang Jain
Allow encoding frames which are directly handed over to the encoder via a span or vector i.e. a raw frame bytes. Introduce an overloaded EncoderLibJpeg::encode() with libcamera::Span source parameter to achieve this functionality. This makes the libjpeg-encoder a bit flexible for use case such as compressing a thumbnail generated for Exif. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-23android: jpeg: encoder: Use pass-by-value for Exif parameterUmang Jain
Following the reasoning of pass-by-value for libcamera::Span parameters from 90c193f2a700("android: Modify Encoder interface") i.e. they are easy to copy/move/construct, align the Exif parameter passing to the encoder interface in this consistent way. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-10-23android: jpeg: Drop "libcamera::" namespace from functions' argumentsUmang Jain
Drop "libcamera::" from functions' arguments for *.cpp files as they have using namespace libcamera; directive in the beginning. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-10-23libcamera: Declare empty virtual destructors as defaultedLaurent Pinchart
The base class of polymorphic classes is required to declare a destructor. Several of these are empty, and can thus be declared as defaulted. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-21android: post_processor_jpeg: Make |cameraDevice_| constantHirokazu Honda
PostProcessorJpeg doesn't have the ownership of CameraDevice given in the constructor. To clarify it, this makes the member variable constant. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-21android: Modify Encoder interfaceHirokazu Honda
In Encoder::encode(), the |source| argument doesn't have to be a pointer. This replaces its type, const pointer, with const reference as the latter is preferred to the former. libcamera::Span is cheap to construct/copy/move. We should deal with the type as pass-by-value parameter. Therefore this also drops the const reference in the |destination| argument. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-21android: Modify PostProcessor interfaceHirokazu Honda
In PostProcessor::process(), the |source| argument doesn't have to be a pointer. This replaces its type, const pointer, with const reference as the latter is preferred to the former. libcamera::Span is cheap to construct/copy/move. We should deal with the type as pass-by-value parameter. Therefore this also drops the const reference in the |destination| argument. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-20android: Omit extra semicolonsHirokazu Honda
The end semicolons with LOG_DECLARE_CATEGORY and LOG_DEFINE_CATEGORY are unnecessary. Signed-off-by: Hirokazu Honda <hiroh@chromium.org> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-10-16android: jpeg: Port to PostProcessor interfaceUmang Jain
Port the CameraStream's JPEG-encoding bits to PostProcessorJpeg. This encapsulates the encoder and EXIF generation code into the PostProcessorJpeg layer and removes these specifics related to JPEG, from the CameraStream itself. Signed-off-by: Umang Jain <email@uajain.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Change-Id: Id9e6e9b2bec83493a90e5e126298a2bb2ed2232a
2020-10-10android: jpeg: Use LGPL-2.1 licenseUmang Jain
The jpeg components are licensed under GPL-2.0-or-later. This is an oversight and unvoluntary. License them under the LGPL-2.1-or-later as the rest of the camera HAL implementation. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-10-07android: jpeg: exif: Sanitize ASCII strings with utils::toAscii()Umang Jain
Use the newly introduced utils::toAscii() utility to remove all non-ASCII characters for EXIF_FORMAT_ASCII strings. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-10-02src: android: exif: Set the class byte orderingKieran Bingham
The exif object sets the byte ordering on construction, and then during later calls re-states the byte ordering when setting values. It could be argued that this ordering should already be known to the exif library and is redundant, but even so we must provide it. Ensure we are consistent in always using the same byte ordering by setting a private class member to re-use a single value. Reviewed-by: Umang Jain <email@uajain.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-09-29android: jpeg: exif: Set timezone informationUmang Jain
The EXIF specification defines three timezone related tags, namely OffsetTime, OffsetTimeOriginal and OffsetTimeDigitized. However, these are not supported by libexif (as of v0.6.21) hence, carry the tags' positional values in our implementation until we get this support from libexif itself. Since these tags were introduced in EXIF specification v2.31, set the exif version number explicitly too. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-09-29android: jpeg: exif: Pad extra byte only for ASCII format in setString()Umang Jain
The EXIF standard states that EXIF_FORMAT_UNDEFINED shall not be terminated with NULL. The patch implements this particular detail and pad one extra byte for EXIF_FORMAT_ASCII to null-terminate strings. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-09-20android: jpeg: exif: Use reentrant localtime_r()Laurent Pinchart
The std::localtime() function isn't thread-safe, and we have no guarantee whether other threads in the camera service may or may not call it. Replace it with localtime_r(). This requires switching from ctime to time.h, as there is no std::localtime_r() function. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Umang Jain <email@uajain.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-09-10android: jpeg: Support an initial set of EXIF metadata tagsUmang Jain
Create a Exif object with various metadata tags set, just before the encoder starts to encode the frame. The object is passed directly as libcamera::Span<> to make sure EXIF tags can be set in a single place i.e. in CameraDevice and the encoder only has the job to write the data in the final output. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-09-10android: jpeg: Add EXIF infrastructureKieran Bingham
Provide helper classes to utilise the libexif interfaces and link against libexif to support tag additions when creating JPEG images. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-08-06android: Introduce JPEG encodingKieran Bingham
Provide an encoder interface and implement a JPEG encoder using libjpeg. Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
by the pipeline handler, some others * by the IPA module which is in charge of handling, for example, the exposure * time and the frame duration. * * This function computes: * - controls::ExposureTime * - controls::FrameDurationLimits */ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) { ControlInfoMap::Map controls{}; double lineDuration = context_.configuration.sensor.lineDuration.get<std::micro>(); /* * Compute exposure time limits by using line length and pixel rate * converted to microseconds. Use the V4L2_CID_EXPOSURE control to get * exposure min, max and default and convert it from lines to * microseconds. */ const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; int32_t minExposure = v4l2Exposure.min().get<int32_t>() * lineDuration; int32_t maxExposure = v4l2Exposure.max().get<int32_t>() * lineDuration; int32_t defExposure = v4l2Exposure.def().get<int32_t>() * lineDuration; controls[&controls::ExposureTime] = ControlInfo(minExposure, maxExposure, defExposure); /* * Compute the frame duration limits. * * The frame length is computed assuming a fixed line length combined * with the vertical frame sizes. */ const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; uint32_t hblank = v4l2HBlank.def().get<int32_t>(); uint32_t lineLength = sensorInfo.outputSize.width + hblank; const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; std::array<uint32_t, 3> frameHeights{ v4l2VBlank.min().get<int32_t>() + sensorInfo.outputSize.height, v4l2VBlank.max().get<int32_t>() + sensorInfo.outputSize.height, v4l2VBlank.def().get<int32_t>() + sensorInfo.outputSize.height, }; std::array<int64_t, 3> frameDurations; for (unsigned int i = 0; i < frameHeights.size(); ++i) { uint64_t frameSize = lineLength * frameHeights[i]; frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); } controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], frameDurations[1], frameDurations[2]); *ipaControls = ControlInfoMap(std::move(controls), controls::controls); } /** * \brief Validate that the sensor controls mandatory for the IPA exists */ bool IPAIPU3::validateSensorControls() { static const uint32_t ctrls[] = { V4L2_CID_ANALOGUE_GAIN, V4L2_CID_EXPOSURE, V4L2_CID_VBLANK, }; for (auto c : ctrls) { if (sensorCtrls_.find(c) == sensorCtrls_.end()) { LOG(IPAIPU3, Error) << "Unable to find sensor control " << utils::hex(c); return false; } } return true; } /** * \brief Initialize the IPA module and its controls * * This function receives the camera sensor information from the pipeline * handler, computes the limits of the controls it handles and returns * them in the \a ipaControls output parameter. */ int IPAIPU3::init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) { camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); if (camHelper_ == nullptr) { LOG(IPAIPU3, Error) << "Failed to create camera sensor helper for " << settings.sensorModel; return -ENODEV; } /* Clean context */ context_.configuration = {}; context_.configuration.sensor.lineDuration = sensorInfo.lineLength * 1.0s / sensorInfo.pixelRate; /* Load the tuning data file. */ File file(settings.configurationFile.c_str()); if (!file.open(File::OpenModeFlag::ReadOnly)) { int ret = file.error(); LOG(IPAIPU3, Error) << "Failed to open configuration file " << settings.configurationFile << ": " << strerror(-ret); return ret; } std::unique_ptr<libcamera::YamlObject> data = YamlParser::parse(file); if (!data) return -EINVAL; unsigned int version = (*data)["version"].get<uint32_t>(0); if (version != 1) { LOG(IPAIPU3, Error) << "Invalid tuning file version " << version; return -EINVAL; } if (!data->contains("algorithms")) { LOG(IPAIPU3, Error) << "Tuning file doesn't contain any algorithm"; return -EINVAL; } int ret = createAlgorithms(context_, (*data)["algorithms"]); if (ret) return ret; /* Initialize controls. */ updateControls(sensorInfo, sensorControls, ipaControls); return 0; } /** * \brief Perform any processing required before the first frame */ int IPAIPU3::start() { /* * Set the sensors V4L2 controls before the first frame to ensure that * we have an expected and known configuration from the start. */ setControls(0); return 0; } /** * \brief Ensure that all processing has completed */ void IPAIPU3::stop() { } /** * \brief Calculate a grid for the AWB statistics * * This function calculates a grid for the AWB algorithm in the IPU3 firmware. * Its input is the BDS output size calculated in the ImgU. * It is limited for now to the simplest method: find the lesser error * with the width/height and respective log2 width/height of the cells. * * \todo The frame is divided into cells which can be 8x8 => 64x64. * As a smaller cell improves the algorithm precision, adapting the * x_start and y_start parameters of the grid would provoke a loss of * some pixels but would also result in more accurate algorithms. */ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) { Size best; Size bestLog2; /* Set the BDS output size in the IPAConfiguration structure */ context_.configuration.grid.bdsOutputSize = bdsOutputSize; uint32_t minError = std::numeric_limits<uint32_t>::max(); for (uint32_t shift = kMinCellSizeLog2; shift <= kMaxCellSizeLog2; ++shift) { uint32_t width = std::clamp(bdsOutputSize.width >> shift, kMinGridWidth, kMaxGridWidth); width = width << shift; uint32_t error = utils::abs_diff(width, bdsOutputSize.width); if (error >= minError) continue; minError = error; best.width = width; bestLog2.width = shift; } minError = std::numeric_limits<uint32_t>::max(); for (uint32_t shift = kMinCellSizeLog2; shift <= kMaxCellSizeLog2; ++shift) { uint32_t height = std::clamp(bdsOutputSize.height >> shift, kMinGridHeight, kMaxGridHeight); height = height << shift; uint32_t error = utils::abs_diff(height, bdsOutputSize.height); if (error >= minError) continue; minError = error; best.height = height; bestLog2.height = shift; } struct ipu3_uapi_grid_config &bdsGrid = context_.configuration.grid.bdsGrid; bdsGrid.x_start = 0; bdsGrid.y_start = 0; bdsGrid.width = best.width >> bestLog2.width; bdsGrid.block_width_log2 = bestLog2.width; bdsGrid.height = best.height >> bestLog2.height; bdsGrid.block_height_log2 = bestLog2.height; /* The ImgU pads the lines to a multiple of 4 cells. */ context_.configuration.grid.stride = utils::alignUp(bdsGrid.width, 4); LOG(IPAIPU3, Debug) << "Best grid found is: (" << (int)bdsGrid.width << " << " << (int)bdsGrid.block_width_log2 << ") x (" << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } /** * \brief Configure the IPU3 IPA * \param[in] configInfo The IPA configuration data, received from the pipeline * handler * \param[in] ipaControls The IPA controls to update * * Calculate the best grid for the statistics based on the pipeline handler BDS * output, and parse the minimum and maximum exposure and analogue gain control * values. * * \todo Document what the BDS is, ideally in a block diagram of the ImgU. * * All algorithm modules are called to allow them to prepare the * \a IPASessionConfiguration structure for the \a IPAContext. */ int IPAIPU3::configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) { if (configInfo.sensorControls.empty()) { LOG(IPAIPU3, Error) << "No sensor controls provided"; return -ENODATA; } sensorInfo_ = configInfo.sensorInfo; lensCtrls_ = configInfo.lensControls; /* * Compute the sensor V4L2 controls to be used by the algorithms and * to be set on the sensor. */ sensorCtrls_ = configInfo.sensorControls; calculateBdsGrid(configInfo.bdsOutputSize); /* Clean IPAActiveState at each reconfiguration. */ context_.activeState = {}; IPAFrameContext initFrameContext; context_.frameContexts.fill(initFrameContext); if (!validateSensorControls()) { LOG(IPAIPU3, Error) << "Sensor control validation failed."; return -EINVAL; } /* Update the camera controls using the new sensor settings. */ updateControls(sensorInfo_, sensorCtrls_, ipaControls); /* Update the IPASessionConfiguration using the sensor settings. */ updateSessionConfiguration(sensorCtrls_); for (auto const &algo : algorithms()) { int ret = algo->configure(context_, configInfo); if (ret) return ret; } return 0; } /** * \brief Map the parameters and stats buffers allocated in the pipeline handler * \param[in] buffers The buffers to map */ void IPAIPU3::mapBuffers(const std::vector<IPABuffer> &buffers) { for (const IPABuffer &buffer : buffers) { const FrameBuffer fb(buffer.planes); buffers_.emplace(buffer.id, MappedFrameBuffer(&fb, MappedFrameBuffer::MapFlag::ReadWrite)); } } /** * \brief Unmap the parameters and stats buffers * \param[in] ids The IDs of the buffers to unmap */ void IPAIPU3::unmapBuffers(const std::vector<unsigned int> &ids) { for (unsigned int id : ids) { auto it = buffers_.find(id); if (it == buffers_.end()) continue; buffers_.erase(it); } } /** * \brief Fill and return a buffer with ISP processing parameters for a frame * \param[in] frame The frame number * \param[in] bufferId ID of the parameter buffer to fill * * Algorithms are expected to fill the IPU3 parameter buffer for the next * frame given their most recent processing of the ImgU statistics. */ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { auto it = buffers_.find(bufferId); if (it == buffers_.end()) { LOG(IPAIPU3, Error) << "Could not find param buffer!"; return; } Span<uint8_t> mem = it->second.planes()[0]; ipu3_uapi_params *params = reinterpret_cast<ipu3_uapi_params *>(mem.data()); /* * The incoming params buffer may contain uninitialised data, or the * parameters of previously queued frames. Clearing the entire buffer * may be an expensive operation, and the kernel will only read from * structures which have their associated use-flag set. * * It is the responsibility of the algorithms to set the use flags * accordingly for any data structure they update during prepare(). */ params->use = {}; for (auto const &algo : algorithms()) algo->prepare(context_, params); paramsBufferReady.emit(frame); } /** * \brief Process the statistics generated by the ImgU * \param[in] frame The frame number * \param[in] frameTimestamp Timestamp of the frame * \param[in] bufferId ID of the statistics buffer * \param[in] sensorControls Sensor controls * * Parse the most recently processed image statistics from the ImgU. The * statistics are passed to each algorithm module to run their calculations and * update their state accordingly. */ void IPAIPU3::processStatsBuffer(const uint32_t frame, [[maybe_unused]] const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) { auto it = buffers_.find(bufferId); if (it == buffers_.end()) { LOG(IPAIPU3, Error) << "Could not find stats buffer!"; return; } Span<uint8_t> mem = it->second.planes()[0]; const ipu3_uapi_stats_3a *stats = reinterpret_cast<ipu3_uapi_stats_3a *>(mem.data()); IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; if (frameContext.frame != frame) LOG(IPAIPU3, Warning) << "Frame " << frame << " does not match its frame context"; frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get<int32_t>(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get<int32_t>()); double lineDuration = context_.configuration.sensor.lineDuration.get<std::micro>(); int32_t vBlank = context_.configuration.sensor.defVBlank; ControlList ctrls(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, &frameContext, stats); setControls(frame); /* \todo Use VBlank value calculated from each frame exposure. */ int64_t frameDuration = (vBlank + sensorInfo_.outputSize.height) * lineDuration; ctrls.set(controls::FrameDuration, frameDuration); ctrls.set(controls::AnalogueGain, frameContext.sensor.gain); ctrls.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); ctrls.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); /* * \todo The Metadata provides a path to getting extended data * out to the application. Further data such as a simplifed Histogram * might have value to be exposed, however such data may be * difficult to report in a generically parsable way and we * likely want to avoid putting platform specific metadata in. */ metadataReady.emit(frame, ctrls); } /** * \brief Queue a request and process the control list from the application * \param[in] frame The number of the frame which will be processed next * \param[in] controls The controls for the \a frame * * Parse the request to handle any IPA-managed controls that were set from the * application such as manual sensor settings. */ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ context_.frameContexts[frame % kMaxFrameContexts] = { frame, controls }; } /** * \brief Handle sensor controls for a given \a frame number * \param[in] frame The frame on which the sensor controls should be set * * Send the desired sensor control values to the pipeline handler to request * that they are applied on the camera sensor. */ void IPAIPU3::setControls(unsigned int frame) { int32_t exposure = context_.activeState.agc.exposure; int32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); ControlList ctrls(sensorCtrls_); ctrls.set(V4L2_CID_EXPOSURE, exposure); ctrls.set(V4L2_CID_ANALOGUE_GAIN, gain); ControlList lensCtrls(lensCtrls_); lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, static_cast<int32_t>(context_.activeState.af.focus)); setSensorControls.emit(frame, ctrls, lensCtrls); } } /* namespace ipa::ipu3 */ /** * \brief External IPA module interface * * The IPAModuleInfo is required to match an IPA module construction against the * intented pipeline handler with the module. The API and pipeline handler * versions must match the corresponding IPA interface and pipeline handler. * * \sa struct IPAModuleInfo */ extern "C" { const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 1, "PipelineHandlerIPU3", "ipu3", }; /** * \brief Create an instance of the IPA interface * * This function is the entry point of the IPA module. It is called by the IPA * manager to create an instance of the IPA interface for each camera. When * matched against with a pipeline handler, the IPAManager will construct an IPA * instance for each associated Camera. */ IPAInterface *ipaCreate() { return new ipa::ipu3::IPAIPU3(); } } } /* namespace libcamera */