summaryrefslogtreecommitdiff
path: root/src/qcam
AgeCommit message (Collapse)Author
2020-06-06libcamera: Rename pixelformats.{cpp,h} to pixel_format.{cpp,h}Laurent Pinchart
The libcamera source files are named after class names, using snake_case. pixelformats.h and pixelformats.cpp don't comply with that rule. Fix them. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-05-28qcam: viewfinder: Use correct DRM/QImage mappingsKieran Bingham
When the native pixel formats supported by QT were introduced, the RGB/BGR formats were inverted. Swap the BGR888 and RGB888 mappings accordingly. Fixes: f890a57b7a06 ("qcam: viewfinder: Add support for more native formats") Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-18(q)cam: Fix header guardsLaurent Pinchart
Several headers belonging to cam and qcam use __LIBCAMERA_*_H__ as a header guard. They're not part of the libcamera core, use __CAM_*_H__ and __QCAM_*_H__ instead, similarly to all other headers of cam and qcam. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-05-13licenses: License all meson files under CC0-1.0Laurent Pinchart
In an attempt to clarify the license terms of all files in the libcamera project, the build system files deserve particular attention. While they describe how the binaries are created, they are not themselves transformed into any part of binary distributions of the software, and thus don't influence the copyright on the binary packages. They are however subject to copyright, and thus influence the distribution terms of the source packages. Most of the meson.build files would not meet the threshold of originality criteria required for copyright protection. Some of the more complex meson.build files may be eligible for copyright protection. To avoid any ambiguity and uncertainty, state our intent to not assert copyrights on the build system files by putting them in the public domain with the CC0-1.0 license. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Acked-by: Giulio Benetti <giulio.benetti@micronovasrl.com> Acked-by: Jacopo Mondi <jacopo@jmondi.org> Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Acked-by: Naushir Patuck <naush@raspberrypi.com> Acked-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> Acked-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Acked-by: Paul Elder <paul.elder@ideasonboard.com> Acked-by: Show Liu <show.liu@linaro.org>
2020-05-04qcam: dng_writer: Write EXIF IFD as custom directoryLaurent Pinchart
The EXIF IFD is incorrectly chained to IFD 0 in addition to being a referenced as a sub IFD through the EXIFIFD tag. While the libtiff API doesn't clearly document why this happens, inspection of the TIFFWriteDirectory() source code show that the function treats the IFD being written as containing an image, which isn't correct for the EXIF IFD. Use TIFFWriteCustomDirectory() instead, which fixes the problem. The resulting DNG file can now be opened with darktable in addition to rawtherapee. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-05-03qcam: dng_writer: Remove colon from \todoNiklas Söderlund
Todo statements should not end with a colon, remove it. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-03qcam: dng_writer: Generate thumbnail in RGB formatLaurent Pinchart
While the DNG specification supports greyscale ("BlackIsZero") for thumbnails, RawTherapee seems to have trouble reading them. Generate thumbnails in RGB instead (but still with greyscale content, to avoid having to interpolate colour components). Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-05-03qcam: dng_writer: Populate DNG tags from metadataLaurent Pinchart
Populate the DNG black level, ISO speed rating and exposure time from metadata. The ISO speed rating and exposure time are standardized as EXIF tags, not TIFF tags, and require a separate EXIF IFD. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-05-03qcam: dng_writer: Output thumbnailLaurent Pinchart
Generate a greyscale, 1/16 resolution thumbnail and add it to the DNG file. This requires shuffling the RAW image generation as the thumbnail has to be stored in the main IFD as per the DNG and TIFF/EP specifications. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-05-03qcam: dng_writer: Name arguments to packScanline()Laurent Pinchart
Name arguments to the FormatInfo::packScanline function pointer to make it easier to understand its usage when reading the function declaration. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-05-03qcam: Pass request metadata to DNG writerLaurent Pinchart
The DNG writer will use the metadata to populate DNG tags. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-05-03qcam: dng_writer: Fix missing field nameNiklas Söderlund
While reformatting the table the field name was missed for one entry, add it. Fixes: db7235b7141aa4e2 ("qcam: Add DNGWriter") Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-02qcam: Add RAW capture supportNiklas Söderlund
Add a toolbar button that captures RAW data to disk. The button is only enabled if the camera is configured to provide a raw stream to the application. Only when the capture action is triggered will a request with a raw buffer be queued to the camera. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-02qcam: Add DNGWriterNiklas Söderlund
Add an initial DNG file writer. The writer can only deal with a small set of pixel formats. The generated file is consumable by standard tools. The writer needs to be extended to write more metadata to the generated file. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-02qcam: Allow for a second raw stream to be configuredNiklas Söderlund
Allow a second stream to be configured for raw capture. This change only adds support for configuring and allocating buffers for the second stream. Later changes are needed to queue the allocated buffers to the camera when the user wishes to capture a raw frame. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-01qcam: Check that camera can generate configuration from rolesNiklas Söderlund
If the camera can not generate a configuration from the requested roles it returns a nullptr which leads to a nullptr dereference. Fix this by adding a check that the camera generated a configuration before trying to access it. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-05-01qcam: Make use of StreamKeyValueParserNiklas Söderlund
Use the StreamKeyValueParser helper to parse stream configuration from the command line. This extends qcam to accept role hints and pixel format in addition to a size. Currently only one viewfinder stream is supported, add a check to keep this behavior. Going forward this restriction will be lifted to support more then one stream. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-04-30qcam: main_window: Fix combo-box entry selection on startupUmang Jain
When one of the camera is selected and opened from the "Select Cameras" items list, the entry of the combo-box in the main-window doesn't update its item index to reflect the camera which was earlier selected. Fix that. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-04-30qcam: main_window: Make cameraCombo_ privateUmang Jain
This commit introduces no functional changes. This is required so that the combo-box list can be managed conveniently from various private functions in subsequent commit. Signed-off-by: Umang Jain <email@uajain.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-04-30qcam: Fix logging of sequence numberLaurent Pinchart
The sequence number of captured frames is logged to the console with padding to 6 characters to increase readability. The output is however incorrect, as 123 is printed as 00012300000. This is caused by the auto-space feature of QDebug, which inserts a space after every field. This doesn't play well with stream format manipulation, as it ends up padding the automatically inserted space the same way as the previous argument. This is a bug in Qt, work around it by formatting the sequence number manually. Fixes: 494da4467ddf ("qcam: Use Qt qInfo() and qWarning() logging facilities") Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-04-27qcam: Don't crash if camera can't be openedLaurent Pinchart
If the camera specified on the command line can't be opened, the MainWindow constructor still proceeds to check the startStopAction_, which results in MainWindow::startCapture() being called and trying to use a null camera_ object. Fix this by returning from the constructor as soon as the error is detected. This also fixes a similar crash if the camera selection dialog box is closed without selecting a camera. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Umang Jain <email@uajain.com>
2020-03-26qcam: Print whole stream configuration when adjustedLaurent Pinchart
When the validate() function adjusts the stream configuration, we print the adjusted size for debugging purpose. Switch to printing the whole configuration, as the pixel format may be useful too. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Prefer stream formats that don't require conversionLaurent Pinchart
Query the viewfinder for the formats it supports natively, and select one of them for the stream if possible. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Report the natively supported pixel formatsLaurent Pinchart
Expose the pixel formats natively supported by the viewfinder, to allow selection of stream formats that would minimize usage of software conversion. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Print message to report format converter usageLaurent Pinchart
Print an info message when initializing the viewfinder to report if the format converter is used or if zero-copy is enabled. This is useful to notify of a possible impact on performances. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Add support for more native formatsLaurent Pinchart
Qt supports more 24-bit and 32-bit RGB formats for native painting. If the frame buffer pixel format matches any of them, disable the converter and create a QImage in the appropriate format. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Display icon when stopping captureLaurent Pinchart
When stopping capture, display an icon instead of the last frame. This is required to be able to release the last buffer when the viewfinder operators in zero-copy mode. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Avoid memory copy when conversion isn't neededLaurent Pinchart
If the frame buffer format is identical to the display format, the viewfinder still invokes the converter to perform what is essentially a slow memcpy(). Make it possible to skip that operation by creating a QImage referencing the buffer memory instead. A reference to the frame buffer is kept internally, and released when the next buffer is queued, pushing the current one out. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Reorder methods to match header fileLaurent Pinchart
Reorder the methods in viewfinder.cpp to match the order in viewfinder.h. No code change. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Use PixelFormat default constructorLaurent Pinchart
There's no need to initialize the PixelFormat stored in ViewFinder explicitly, as PixelFormat is now a class with a default constructor. Remove the initialization. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Embed QImage in ViewFinderLaurent Pinchart
The QImage class is a thin wrapper that uses implicit sharing. We can thus embed it in the ViewFinder class instead of allocating it dynamically, and assign it at runtime. This simplifies the code. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Make the viewfinder hold a reference to a bufferLaurent Pinchart
The viewfinder is currently expected to render frames to the screen synchronously in the display() function, or at least to copy data so that the buffer can be queued in a new request when the function returns. This prevents optimisations when the capture format is identical to the display format. Make the viewfinder take ownership of the buffer, and notify of its release through a signal. The release is currently still synchronous, this will be addressed in a subsequent patch. Rename the ViewFinder::display() function to render() to better describe its purpose, as it's meant to start the rendering and not display the frame synchronously. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Move multi-planar check into viewfinderLaurent Pinchart
The lack of support for multiplanar buffers comes from the viewfinder. Move the corresponding check from MainWindow. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: viewfinder: Add MappedBuffer to store memory mapping informationLaurent Pinchart
The new MappedBuffer structure replaces the std::pair<> used in the mapped buffers map, and allows passing data to the ViewFinder::display() function in a more structured way. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: Use Qt qInfo() and qWarning() logging facilitiesLaurent Pinchart
Replace manual usage of std::cout and std::cerr with the Qt logging facilities. This allows redirection log output if needed, and integrates better with Qt. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Remove unneeded debug messageLaurent Pinchart
Printing the name of the selected camera to the log doesn't provide any value. Remove it. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Document functions and reorganize member dataLaurent Pinchart
The qcam application is our reference implementation of a libcamera GUI application. Document the code of the MainWindow class to make it easier to follow, and reorganize the member data in logical groups for clarity. No code change. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Don't print message when saving a pictureLaurent Pinchart
When saving a picture, the application prints a message on cout. This isn't necessary and doesn't really help with debugging or diagnostics, remove it. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Add shortcuts for toolbar actionsLaurent Pinchart
Allow triggering toolbar actions with keyboard shortcuts. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Use icons from system icon themeLaurent Pinchart
Use the system icon theme by default, falling back to custom icons if no theme is available. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Replace start and stop actions with a toggle actionLaurent Pinchart
The main window toolbar contains a start button and a stop button. This allows starting an already started camera (which is currently not handled and results in an error) or stopping an already stopped camera. Replace the two actions with a single start/stop toggle action, preventing UI misuse and reducing confusion. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Move capture event processing to main threadLaurent Pinchart
To avoid blocking the camera manager for a long amount of time, move capture event processing to the main thread. Captured buffers are added to a queue and an event is posted to the main window to signal availability of a buffer. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: main_window: Move request queuing to a separate functionLaurent Pinchart
Requests are requeued synchronously from the completion handler. To prepare for delayed requeuing, move the queuing to a separate function. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: Use QSize through the code baseLaurent Pinchart
Qt has a QSize class to store sizes. Use it to replace width and height where applicable. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: Ensure headers are self-containedLaurent Pinchart
Include the headers corresponding to each compile unit at the very first line to ensure they are self-contained. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-24qcam: Remove custom event dispatcherLaurent Pinchart
The qcam application installs a custom event dispatcher based on the Qt event loop. As the camera manager now creates an internal thread, it doesn't use that event dispatcher of the application thread at all. Furthermore, the custom event dispatcher is buggy, as it doesn't dispatch messages posted to the main thread's event loop. This isn't an issue as no messages are posted there in the first place, but would cause incorrect behaviour if we were to use that feature (for instance to deliver signals from the camera manager thread to the application thread). Fixing the event dispatcher requires a change in the libcamera public API, as there's currently no way to dispatch messages using the public API (Thread::dispatchMessages() is not exposed). This isn't worth it at the moment, so just remove the custom event dispatcher. If qcam later needs the libcamera request and buffer completion signals to be delivered in the application thread, it will need to handle that internally, using Qt's cross-thread signal delivery. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-19qcam: saveImageAs: Set image quality explicitlyKieran Bingham
The QImageWriter defaults to an image quality value of 75 which can lead to perceivable visual artefacts. Improve the quality of the output by explicitly setting the image quality to 95. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2020-03-18libcamera: framebuffer_allocator: Lift camera restrictions on allocatorLaurent Pinchart
The Camera class currently requires the allocator to have no allocated buffer before the camera is reconfigured, and the allocator to be destroyed before the camera is released. There's no basis for these restrictions anymore, remove them. Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
2020-03-18libcamera: PixelFormat: Mark all function arguments of type PixelFormat as ↵Niklas Söderlund
const reference PixelFormat was previously an alias for unsigned int but is now a class. Make all functions taking PixelFormat do so as a const reference. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2020-03-18libcamera: pixelformats: include linux/drm_fourcc.hNiklas Söderlund
Instead of having to include linux/drm_fourcc.h everywhere a DRM FourCC is used in conjunction with PixelFormat include the header directly in pixelformats.h. Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
@@\n' % \ (Colours.fg(Colours.Cyan), self.__from.start, len(self.__from), self.__to.start, len(self.__to)) for line in self.lines: if line[0] == '-': s += Colours.fg(Colours.Red) elif line[0] == '+': s += Colours.fg(Colours.Green) if line[0] == '-': spaces = 0 for i in range(len(line)): if line[-i-1].isspace(): spaces += 1 else: break spaces = len(line) - spaces line = line[0:spaces] + Colours.bg(Colours.Red) + line[spaces:] s += line s += Colours.reset() s += '\n' return s[:-1] def append(self, line): if line[0] == ' ': self.__from.untouched.append(self.__from_line) self.__from_line += 1 self.__to.untouched.append(self.__to_line) self.__to_line += 1 elif line[0] == '-': self.__from.touched.append(self.__from_line) self.__from_line += 1 elif line[0] == '+': self.__to.touched.append(self.__to_line) self.__to_line += 1 self.lines.append(line.rstrip('\n')) def intersects(self, lines): for line in lines: if line in self.__from.touched: return True return False def side(self, side): if side == 'from': return self.__from else: return self.__to def parse_diff(diff): hunks = [] hunk = None for line in diff: if line.startswith('@@'): if hunk: hunks.append(hunk) hunk = DiffHunk(line) elif hunk is not None: hunk.append(line) if hunk: hunks.append(hunk) return hunks # ------------------------------------------------------------------------------ # Commit, Staged Changes & Amendments # class CommitFile: def __init__(self, name): info = name.split() self.__status = info[0][0] # For renamed files, store the new name if self.__status == 'R': self.__filename = info[2] else: self.__filename = info[1] @property def filename(self): return self.__filename @property def status(self): return self.__status class Commit: def __init__(self, commit): self.commit = commit self._parse() def _parse(self): # Get the commit title and list of files. ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-status', self.commit], stdout=subprocess.PIPE).stdout.decode('utf-8') files = ret.splitlines() self._files = [CommitFile(f) for f in files[1:]] self._title = files[0] def files(self, filter='AMR'): return [f.filename for f in self._files if f.status in filter] @property def title(self): return self._title def get_diff(self, top_level, filename): diff = subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), '--', '%s/%s' % (top_level, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') return parse_diff(diff.splitlines(True)) def get_file(self, filename): return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') class StagedChanges(Commit): def __init__(self): Commit.__init__(self, '') def _parse(self): ret = subprocess.run(['git', 'diff', '--staged', '--name-status'], stdout=subprocess.PIPE).stdout.decode('utf-8') self._title = 'Staged changes' self._files = [CommitFile(f) for f in ret.splitlines()] def get_diff(self, top_level, filename): diff = subprocess.run(['git', 'diff', '--staged', '--', '%s/%s' % (top_level, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') return parse_diff(diff.splitlines(True)) class Amendment(StagedChanges): def __init__(self): StagedChanges.__init__(self) def _parse(self): # Create a title using HEAD commit ret = subprocess.run(['git', 'show', '--pretty=oneline', '--no-patch'], stdout=subprocess.PIPE).stdout.decode('utf-8') self._title = 'Amendment of ' + ret.strip() # Extract the list of modified files ret = subprocess.run(['git', 'diff', '--staged', '--name-status', 'HEAD~'], stdout=subprocess.PIPE).stdout.decode('utf-8') self._files = [CommitFile(f) for f in ret.splitlines()] def get_diff(self, top_level, filename): diff = subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', '%s/%s' % (top_level, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') return parse_diff(diff.splitlines(True)) # ------------------------------------------------------------------------------ # Helpers # class ClassRegistry(type): def __new__(cls, clsname, bases, attrs): newclass = super().__new__(cls, clsname, bases, attrs) if bases: bases[0].subclasses.append(newclass) return newclass # ------------------------------------------------------------------------------ # Commit Checkers # class CommitChecker(metaclass=ClassRegistry): subclasses = [] def __init__(self): pass # # Class methods # @classmethod def checkers(cls): for checker in cls.subclasses: yield checker class CommitIssue(object): def __init__(self, msg): self.msg = msg class HeaderAddChecker(CommitChecker): @classmethod def check(cls, commit, top_level): issues = [] meson_files = [f for f in commit.files('M') if os.path.basename(f) == 'meson.build'] for filename in commit.files('AR'): if not filename.startswith('include/libcamera/') or \ not filename.endswith('.h'): continue meson = os.path.dirname(filename) + '/meson.build' header = os.path.basename(filename) issue = CommitIssue('Header %s added without corresponding update to %s' % (filename, meson)) if meson not in meson_files: issues.append(issue) continue diff = commit.get_diff(top_level, meson) found = False for hunk in diff: for line in hunk.lines: if line[0] != '+': continue if line.find("'%s'" % header) != -1: found = True break if found: break if not found: issues.append(issue) return issues # ------------------------------------------------------------------------------ # Style Checkers # class StyleChecker(metaclass=ClassRegistry): subclasses = [] def __init__(self): pass # # Class methods # @classmethod def checkers(cls, filename): for checker in cls.subclasses: if checker.supports(filename): yield checker @classmethod def supports(cls, filename): for pattern in cls.patterns: if fnmatch.fnmatch(os.path.basename(filename), pattern): return True return False @classmethod def all_patterns(cls): patterns = set() for checker in cls.subclasses: patterns.update(checker.patterns) return patterns class StyleIssue(object): def __init__(self, line_number, line, msg): self.line_number = line_number self.line = line self.msg = msg class IncludeChecker(StyleChecker): patterns = ('*.cpp', '*.h') headers = ('assert', 'ctype', 'errno', 'fenv', 'float', 'inttypes', 'limits', 'locale', 'setjmp', 'signal', 'stdarg', 'stddef', 'stdint', 'stdio', 'stdlib', 'string', 'time', 'uchar', 'wchar', 'wctype') include_regex = re.compile('^#include <c([a-z]*)>') def __init__(self, content): super().__init__() self.__content = content def check(self, line_numbers): issues = [] for line_number in line_numbers: line = self.__content[line_number - 1] match = IncludeChecker.include_regex.match(line) if not match: continue header = match.group(1) if header not in IncludeChecker.headers: continue issues.append(StyleIssue(line_number, line, 'C compatibility header <%s.h> is preferred' % header)) return issues class LogCategoryChecker(StyleChecker): log_regex = re.compile('\\bLOG\((Debug|Info|Warning|Error|Fatal)\)') patterns = ('*.cpp',) def __init__(self, content): super().__init__() self.__content = content def check(self, line_numbers): issues = [] for line_number in line_numbers: line = self.__content[line_number-1] if not LogCategoryChecker.log_regex.search(line): continue issues.append(StyleIssue(line_number, line, 'LOG() should use categories')) return issues class MesonChecker(StyleChecker): patterns = ('meson.build',) def __init__(self, content): super().__init__() self.__content = content def check(self, line_numbers): issues = [] for line_number in line_numbers: line = self.__content[line_number-1] if line.find('\t') != -1: issues.append(StyleIssue(line_number, line, 'meson.build should use spaces for indentation')) return issues class Pep8Checker(StyleChecker): patterns = ('*.py',) results_regex = re.compile('stdin:([0-9]+):([0-9]+)(.*)') def __init__(self, content): super().__init__() self.__content = content def check(self, line_numbers): issues = [] data = ''.join(self.__content).encode('utf-8') try: ret = subprocess.run(['pycodestyle', '--ignore=E501', '-'], input=data, stdout=subprocess.PIPE) except FileNotFoundError: issues.append(StyleIssue(0, None, 'Please install pycodestyle to validate python additions')) return issues results = ret.stdout.decode('utf-8').splitlines() for item in results: search = re.search(Pep8Checker.results_regex, item) line_number = int(search.group(1)) position = int(search.group(2)) msg = search.group(3) if line_number in line_numbers: line = self.__content[line_number - 1] issues.append(StyleIssue(line_number, line, msg)) return issues class ShellChecker(StyleChecker): patterns = ('*.sh',) results_line_regex = re.compile('In - line ([0-9]+):') def __init__(self, content): super().__init__() self.__content = content def check(self, line_numbers): issues = [] data = ''.join(self.__content).encode('utf-8') try: ret = subprocess.run(['shellcheck', '-Cnever', '-'], input=data, stdout=subprocess.PIPE) except FileNotFoundError: issues.append(StyleIssue(0, None, 'Please install shellcheck to validate shell script additions')) return issues results = ret.stdout.decode('utf-8').splitlines() for nr, item in enumerate(results): search = re.search(ShellChecker.results_line_regex, item) if search is None: continue line_number = int(search.group(1)) line = results[nr + 1] msg = results[nr + 2] # Determined, but not yet used position = msg.find('^') + 1 if line_number in line_numbers: issues.append(StyleIssue(line_number, line, msg)) return issues # ------------------------------------------------------------------------------ # Formatters # class Formatter(metaclass=ClassRegistry): subclasses = [] def __init__(self): pass # # Class methods # @classmethod def formatters(cls, filename): for formatter in cls.subclasses: if formatter.supports(filename): yield formatter @classmethod def supports(cls, filename): for pattern in cls.patterns: if fnmatch.fnmatch(os.path.basename(filename), pattern): return True return False @classmethod def all_patterns(cls): patterns = set() for formatter in cls.subclasses: patterns.update(formatter.patterns) return patterns class CLangFormatter(Formatter): patterns = ('*.c', '*.cpp', '*.h') @classmethod def format(cls, filename, data): ret = subprocess.run(['clang-format', '-style=file', '-assume-filename=' + filename], input=data.encode('utf-8'), stdout=subprocess.PIPE) return ret.stdout.decode('utf-8') class DoxygenFormatter(Formatter): patterns = ('*.c', '*.cpp') return_regex = re.compile(' +\\* +\\\\return +[a-z]') @classmethod def format(cls, filename, data): lines = [] in_doxygen = False for line in data.split('\n'): if line.find('/**') != -1: in_doxygen = True if not in_doxygen: lines.append(line) continue line = cls.return_regex.sub(lambda m: m.group(0)[:-1] + m.group(0)[-1].upper(), line) if line.find('*/') != -1: in_doxygen = False lines.append(line) return '\n'.join(lines) class DPointerFormatter(Formatter): # Ensure consistent naming of variables related to the d-pointer design # pattern. patterns = ('*.cpp', '*.h') # The clang formatter runs first, we can thus rely on appropriate coding # style. declare_regex = re.compile(r'^(\t*)(const )?([a-zA-Z0-9_]+) \*( ?const )?([a-zA-Z0-9_]+) = (LIBCAMERA_[DO]_PTR)\(([a-zA-Z0-9_]+)\);$') @classmethod def format(cls, filename, data): lines = [] for line in data.split('\n'): match = cls.declare_regex.match(line) if match: indent = match.group(1) or '' const = match.group(2) or '' macro = match.group(6) klass = match.group(7) if macro == 'LIBCAMERA_D_PTR': var = 'Private *const d' else: var = f'{klass} *const o' line = f'{indent}{const}{var} = {macro}({klass});' lines.append(line) return '\n'.join(lines) class IncludeOrderFormatter(Formatter): patterns = ('*.cpp', '*.h') include_regex = re.compile('^#include ["<]([^">]*)[">]') @classmethod def format(cls, filename, data): lines = [] includes = [] # Parse blocks of #include statements, and output them as a sorted list # when we reach a non #include statement. for line in data.split('\n'): match = IncludeOrderFormatter.include_regex.match(line) if match: # If the current line is an #include statement, add it to the # includes group and continue to the next line. includes.append((line, match.group(1))) continue # The current line is not an #include statement, output the sorted # stashed includes first, and then the current line. if len(includes): includes.sort(key=lambda i: i[1]) for include in includes: lines.append(include[0]) includes = [] lines.append(line) # In the unlikely case the file ends with an #include statement, make # sure we output the stashed includes. if len(includes): includes.sort(key=lambda i: i[1]) for include in includes: lines.append(include[0]) includes = [] return '\n'.join(lines) class StripTrailingSpaceFormatter(Formatter): patterns = ('*.c', '*.cpp', '*.h', '*.py', 'meson.build') @classmethod def format(cls, filename, data): lines = data.split('\n') for i in range(len(lines)): lines[i] = lines[i].rstrip() + '\n' return ''.join(lines) # ------------------------------------------------------------------------------ # Style checking # def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. commit_diff = commit.get_diff(top_level, filename) lines = [] for hunk in commit_diff: lines.extend(hunk.side('to').touched) # Skip commits that don't add any line. if len(lines) == 0: return 0 # Format the file after the commit with all formatters and compute the diff # between the unformatted and formatted contents. after = commit.get_file(filename) formatted = after for formatter in Formatter.formatters(filename): formatted = formatter.format(filename, formatted) after = after.splitlines(True) formatted = formatted.splitlines(True) diff = difflib.unified_diff(after, formatted) # Split the diff in hunks, recording line number ranges for each hunk, and # filter out hunks that are not touched by the commit. formatted_diff = parse_diff(diff) formatted_diff = [hunk for hunk in formatted_diff if hunk.intersects(lines)] # Check for code issues not related to formatting. issues = [] for checker in StyleChecker.checkers(filename): checker = checker(after) for hunk in commit_diff: issues += checker.check(hunk.side('to').touched) # Print the detected issues. if len(issues) == 0 and len(formatted_diff) == 0: return 0 print('%s---' % Colours.fg(Colours.Red), filename) print('%s+++' % Colours.fg(Colours.Green), filename) if len(formatted_diff): for hunk in formatted_diff: print(hunk) if len(issues): issues = sorted(issues, key=lambda i: i.line_number) for issue in issues: print('%s#%u: %s' % (Colours.fg(Colours.Yellow), issue.line_number, issue.msg)) if issue.line is not None: print('+%s%s' % (issue.line.rstrip(), Colours.reset())) return len(formatted_diff) + len(issues) def check_style(top_level, commit): separator = '-' * len(commit.title) print(separator) print(commit.title) print(separator) issues = 0 # Apply the commit checkers first. for checker in CommitChecker.checkers(): for issue in checker.check(commit, top_level): print('%s%s%s' % (Colours.fg(Colours.Yellow), issue.msg, Colours.reset())) issues += 1 # Filter out files we have no checker for. patterns = set() patterns.update(StyleChecker.all_patterns()) patterns.update(Formatter.all_patterns()) files = [f for f in commit.files() if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] for f in files: issues += check_file(top_level, commit, f) if issues == 0: print('No issue detected') else: print('---') print('%u potential %s detected, please review' % (issues, 'issue' if issues == 1 else 'issues')) return issues def extract_commits(revs): """Extract a list of commits on which to operate from a revision or revision range. """ ret = subprocess.run(['git', 'rev-parse', revs], stdout=subprocess.PIPE, stderr=subprocess.PIPE) if ret.returncode != 0: print(ret.stderr.decode('utf-8').splitlines()[0]) return [] revlist = ret.stdout.decode('utf-8').splitlines() # If the revlist contains more than one item, pass it to git rev-list to list # each commit individually. if len(revlist) > 1: ret = subprocess.run(['git', 'rev-list', *revlist], stdout=subprocess.PIPE) revlist = ret.stdout.decode('utf-8').splitlines() revlist.reverse() return [Commit(x) for x in revlist] def git_top_level(): """Get the absolute path of the git top-level directory.""" ret = subprocess.run(['git', 'rev-parse', '--show-toplevel'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) if ret.returncode != 0: print(ret.stderr.decode('utf-8').splitlines()[0]) return None return ret.stdout.decode('utf-8').strip() def main(argv): # Parse command line arguments parser = argparse.ArgumentParser() parser.add_argument('--staged', '-s', action='store_true', help='Include the changes in the index. Defaults to False') parser.add_argument('--amend', '-a', action='store_true', help='Include changes in the index and the previous patch combined. Defaults to False') parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) # Check for required dependencies. for command, mandatory in dependencies.items(): found = shutil.which(command) if mandatory and not found: print('Executable %s not found' % command) return 1 dependencies[command] = found # Get the top level directory to pass absolute file names to git diff # commands, in order to support execution from subdirectories of the git # tree. top_level = git_top_level() if top_level is None: return 1 commits = [] if args.staged: commits.append(StagedChanges()) if args.amend: commits.append(Amendment()) # If none of --staged or --amend was passed if len(commits) == 0: # And no revisions were passed, then default to HEAD if not args.revision_range: args.revision_range = 'HEAD' if args.revision_range: commits += extract_commits(args.revision_range) issues = 0 for commit in commits: issues += check_style(top_level, commit) print('') if issues: return 1 else: return 0 if __name__ == '__main__': sys.exit(main(sys.argv))