GNU LESSER GENERAL PUBLIC LICENSE Version 2.1, February 1999 Copyright (C) 1991, 1999 Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. [This is the first released version of the Lesser GPL. It also counts as the successor of the GNU Library Public License, version 2, hence the version number 2.1.] Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public Licenses are intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This license, the Lesser General Public License, applies to some specially designated software packages--typically libraries--of the Free Software Foundation and other authors who decide to use it. You can use it too, but we suggest you first think carefully about whether this license or the ordinary General Public License is the better strategy to use in any particular case, based on the explanations below. When we speak of free software, we are referring to freedom of use, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish); that you receive source code or can get it if you want it; that you can change the software and use pieces of it in new free programs; and that you are informed that you can do these things. To protect your rights, we need to make restrictions that forbid distributors to deny you these rights or to ask you to surrender these rights. These restrictions translate to certain responsibilities for you if you distribute copies of the library or if you modify it. For example, if you distribute copies of the library, whether gratis or for a fee, you must give the recipients all the rights that we gave you. You must make sure that they, too, receive or can get the source code. If you link other code with the library, you must provide complete object files to the recipients, so that they can relink them with the library after making changes to the library and recompiling it. And you must show them these terms so they know their rights. We protect your rights with a two-step method: (1) we copyright the library, and (2) we offer you this license, which gives you legal permission to copy, distribute and/or modify the library. To protect each distributor, we want to make it very clear that there is no warranty for the free library. Also, if the library is modified by someone else and passed on, the recipients should know that what they have is not the original version, so that the original author's reputation will not be affected by problems that might be introduced by others. Finally, software patents pose a constant threat to the existence of any free program. We wish to make sure that a company cannot effectively restrict the users of a free program by obtaining a restrictive license from a patent holder. Therefore, we insist that any patent license obtained for a version of the library must be consistent with the full freedom of use specified in this license. Most GNU software, including some libraries, is covered by the ordinary GNU General Public License. This license, the GNU Lesser General Public License, applies to certain designated libraries, and is quite different from the ordinary General Public License. We use this license for certain libraries in order to permit linking those libraries into non-free programs. When a program is linked with a library, whether statically or using a shared library, the combination of the two is legally speaking a combined work, a derivative of the original library. The ordinary General Public License therefore permits such linking only if the entire combination fits its criteria of freedom. The Lesser General Public License permits more lax criteria for linking other code with the library. We call this license the "Lesser" General Public License because it does Less to protect the user's freedom than the ordinary General Public License. It also provides other free software developers Less of an advantage over competing non-free programs. These disadvantages are the reason we use the ordinary General Public License for many libraries. However, the Lesser license provides advantages in certain special circumstances. For example, on rare occasions, there may be a special need to encourage the widest possible use of a certain library, so that it becomes a de-facto standard. To achieve this, non-free programs must be allowed to use the library. A more frequent case is that a free library does the same job as widely used non-free libraries. In this case, there is little to gain by limiting the free library to free software only, so we use the Lesser General Public License. In other cases, permission to use a particular library in non-free programs enables a greater number of people to use a large body of free software. For example, permission to use the GNU C Library in non-free programs enables many more people to use the whole GNU operating system, as well as its variant, the GNU/Linux operating system. Although the Lesser General Public License is Less protective of the users' freedom, it does ensure that the user of a program that is linked with the Library has the freedom and the wherewithal to run that program using a modified version of the Library. The precise terms and conditions for copying, distribution and modification follow. Pay close attention to the difference between a "work based on the library" and a "work that uses the library". The former contains code derived from the library, whereas the latter must be combined with the library in order to run. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License Agreement applies to any software library or other program which contains a notice placed by the copyright holder or other authorized party saying it may be distributed under the terms of this Lesser General Public License (also called "this License"). Each licensee is addressed as "you". A "library" means a collection of software functions and/or data prepared so as to be conveniently linked with application programs (which use some of those functions and data) to form executables. The "Library", below, refers to any such software library or work which has been distributed under these terms. A "work based on the Library" means either the Library or any derivative work under copyright law: that is to say, a work containing the Library or a portion of it, either verbatim or with modifications and/or translated straightforwardly into another language. (Hereinafter, translation is included without limitation in the term "modification".) "Source code" for a work means the preferred form of the work for making modifications to it. For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library. Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running a program using the Library is not restricted, and output from such a program is covered only if its contents constitute a work based on the Library (independent of the use of the Library in a tool for writing it). Whether that is true depends on what the Library does and what the program that uses the Library does. 1. You may copy and distribute verbatim copies of the Library's complete source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and distribute a copy of this License along with the Library. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Library or any portion of it, thus forming a work based on the Library, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) The modified work must itself be a software library. b) You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change. c) You must cause the whole of the work to be licensed at no charge to all third parties under the terms of this License. d) If a facility in the modified Library refers to a function or a table of data to be supplied by an application program that uses the facility, other than as an argument passed when the facility is invoked, then you must make a good faith effort to ensure that, in the event an application does not supply such function or table, the facility still operates, and performs whatever part of its purpose remains meaningful. (For example, a function in a library to compute square roots has a purpose that is entirely well-defined independent of the application. Therefore, Subsection 2d requires that any application-supplied function or table used by this function must be optional: if the application does not supply it, the square root function must still compute square roots.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Library, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Library, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Library. In addition, mere aggregation of another work not based on the Library with the Library (or with a work based on the Library) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may opt to apply the terms of the ordinary GNU General Public License instead of this License to a given copy of the Library. To do this, you must alter all the notices that refer to this License, so that they refer to the ordinary GNU General Public License, version 2, instead of to this License. (If a newer version than version 2 of the ordinary GNU General Public License has appeared, then you can specify that version instead if you wish.) Do not make any other change in these notices. Once this change is made in a given copy, it is irreversible for that copy, so the ordinary GNU General Public License applies to all subsequent copies and derivative works made from that copy. This option is useful when you wish to copy part of the code of the Library into a program that is not a library. 4. You may copy and distribute the Library (or a portion or derivative of it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange. If distribution of object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place satisfies the requirement to distribute the source code, even though third parties are not compelled to copy the source along with the object code. 5. A program that contains no derivative of any portion of the Library, but is designed to work with the Library by being compiled or linked with it, is called a "work that uses the Library". Such a work, in isolation, is not a derivative work of the Library, and therefore falls outside the scope of this License. However, linking a "work that uses the Library" with the Library creates an executable that is a derivative of the Library (because it contains portions of the Library), rather than a "work that uses the library". The executable is therefore covered by this License. Section 6 states terms for distribution of such executables. When a "work that uses the Library" uses material from a header file that is part of the Library, the object code for the work may be a derivative work of the Library even though the source code is not. Whether this is true is especially significant if the work can be linked without the Library, or if the work is itself a library. The threshold for this to be true is not precisely defined by law. If such an object file uses only numerical parameters, data structure layouts and accessors, and small macros and small inline functions (ten lines or less in length), then the use of the object file is unrestricted, regardless of whether it is legally a derivative work. (Executables containing this object code plus portions of the Library will still fall under Section 6.) Otherwise, if the work is a derivative of the Library, you may distribute the object code for the work under the terms of Section 6. Any executables containing that work also fall under Section 6, whether or not they are linked directly with the Library itself. 6. As an exception to the Sections above, you may also combine or link a "work that uses the Library" with the Library to produce a work containing portions of the Library, and distribute that work under terms of your choice, provided that the terms permit modification of the work for the customer's own use and reverse engineering for debugging such modifications. You must give prominent notice with each copy of the work that the Library is used in it and that the Library and its use are covered by this License. You must supply a copy of this License. If the work during execution displays copyright notices, you must include the copyright notice for the Library among them, as well as a reference directing the user to the copy of this License. Also, you must do one of these things: a) Accompany the work with the complete corresponding machine-readable source code for the Library including whatever changes were used in the work (which must be distributed under Sections 1 and 2 above); and, if the work is an executable linked with the Library, with the complete machine-readable "work that uses the Library", as object code and/or source code, so that the user can modify the Library and then relink to produce a modified executable containing the modified Library. (It is understood that the user who changes the contents of definitions files in the Library will not necessarily be able to recompile the application to use the modified definitions.) b) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (1) uses at run time a copy of the library already present on the user's computer system, rather than copying library functions into the executable, and (2) will operate properly with a modified version of the library, if the user installs one, as long as the modified version is interface-compatible with the version that the work was made with. c) Accompany the work with a written offer, valid for at least three years, to give the same user the materials specified in Subsection 6a, above, for a charge no more than the cost of performing this distribution. d) If distribution of the work is made by offering access to copy from a designated place, offer equivalent access to copy the above specified materials from the same place. e) Verify that the user has already received a copy of these materials or that you have already sent this user a copy. For an executable, the required form of the "work that uses the Library" must include any data and utility programs needed for reproducing the executable from it. However, as a special exception, the materials to be distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. It may happen that this requirement contradicts the license restrictions of other proprietary libraries that do not normally accompany the operating system. Such a contradiction means you cannot use both them and the Library together in an executable that you distribute. 7. You may place library facilities that are a work based on the Library side-by-side in a single library together with other library facilities not covered by this License, and distribute such a combined library, provided that the separate distribution of the work based on the Library and of the other library facilities is otherwise permitted, and provided that you do these two things: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities. This must be distributed under the terms of the Sections above. b) Give prominent notice with the combined library of the fact that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 8. You may not copy, modify, sublicense, link with, or distribute the Library except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, link with, or distribute the Library is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 9. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Library or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Library (or any work based on the Library), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Library or works based on it. 10. Each time you redistribute the Library (or any work based on the Library), the recipient automatically receives a license from the original licensor to copy, distribute, link with or modify the Library subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties with this License. 11. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Library at all. For example, if a patent license would not permit royalty-free redistribution of the Library by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Library. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply, and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 12. If the distribution and/or use of the Library is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Library under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 13. The Free Software Foundation may publish revised and/or new versions of the Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Library does not specify a license version number, you may choose any version ever published by the Free Software Foundation. 14. If you wish to incorporate parts of the Library into other free programs whose distribution conditions are incompatible with these, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Libraries If you develop a new library, and you want it to be of the greatest possible use to the public, we recommend making it free software that everyone can redistribute and change. You can do so by permitting redistribution under these terms (or, alternatively, under the terms of the ordinary General Public License). To apply these terms, attach the following notices to the library. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Also add information on how to contact you by electronic and paper mail. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the library, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the library `Frob' (a library for tweaking knobs) written by James Random Hacker. < signature of Ty Coon > , 1 April 1990 Ty Coon, President of Vice That's all there is to it! 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 
.. SPDX-License-Identifier: CC-BY-SA-4.0

Pipeline Handler Writers Guide
==============================

Pipeline handlers are the abstraction layer for device-specific hardware
configuration. They access and control hardware through the V4L2 and Media
Controller kernel interfaces, and implement an internal API to control the ISP
and capture components of a pipeline directly.

Prerequisite knowledge: system architecture
-------------------------------------------

A pipeline handler configures and manages the image acquisition and
transformation pipeline realized by specialized system peripherals combined with
an image source connected to the system through a data and control bus. The
presence, number and characteristics of them vary depending on the system design
and the product integration of the target platform.

System components can be classified in three macro-categories:

.. TODO: Insert references to the open CSI-2 (and other) specification.

- Input ports: Interfaces to external devices, usually image sensors,
  which transfer data from the physical bus to locations accessible by other
  system peripherals. An input port needs to be configured according to the
  input image format and size and could optionally apply basic transformations
  on the received images, most typically cropping/scaling and some formats
  conversion. The industry standard for the system typically targeted by
  libcamera is to have receivers compliant with the MIPI CSI-2 specifications,
  implemented on a compatible physical layer such as MIPI D-PHY or MIPI C-PHY.
  Other design are possible but less common, such as LVDS or the legacy BT.601
  and BT.656 parallel protocols.

- Image Signal Processor (ISP): A specialized media processor which applies
  digital transformations on image streams. ISPs can be integrated as part of
  the SoC as a memory interfaced system peripheral or packaged as stand-alone
  chips connected to the application processor through a bus. Most hardware used
  by libcamera makes use of in-system ISP designs but pipelines can equally
  support external ISP chips or be instrumented to use other system resources
  such as a GPU or an FPGA IP block. ISPs expose a software programming
  interface that allows the configuration of multiple processing blocks which
  form an "Image Transformation Pipeline". An ISP usually produces 'processed'
  image streams along with the metadata describing the processing steps which
  have been applied to generate the output frames.

- Camera Sensor: Digital components that integrate an image sensor with control
  electronics and usually a lens. It interfaces to the SoC image receiver ports
  and is programmed to produce images in a format and size suitable for the
  current system configuration. Complex camera modules can integrate on-board
  ISP or DSP chips and process images before delivering them to the system. Most
  systems with a dedicated ISP processor will usually integrate camera sensors
  which produce images in Raw Bayer format and defer processing to it.

It is the responsibility of the pipeline handler to interface with these (and
possibly other) components of the system and implement the following
functionalities:

- Detect and register camera devices available in the system with an associated
  set of image streams.

- Configure the image acquisition and processing pipeline by assigning the
  system resources (memory, shared components, etc.) to satisfy the
  configuration requested by the application.

- Start and stop the image acquisition and processing sessions.

- Apply configuration settings requested by applications and computed by image
  processing algorithms integrated in libcamera to the hardware devices.

- Notify applications of the availability of new images and deliver them to the
  correct locations.

Prerequisite knowledge: libcamera architecture
----------------------------------------------

A pipeline handler makes use of the following libcamera classes to realize the
functionalities descibed above. Below is a brief overview of each of those:

.. TODO: (All) Convert to sphinx refs
.. TODO: (MediaDevice) Reference to the Media Device API (possibly with versioning requirements)
.. TODO: (IPAInterface) refer to the IPA guide

-  `MediaDevice <http://libcamera.org/api-html/classlibcamera_1_1MediaDevice.html>`_:
   Instances of this class are associated with a kernel media controller
   device and its connected objects.

-  `DeviceEnumerator <http://libcamera.org/api-html/classlibcamera_1_1DeviceEnumerator.html>`_:
   Enumerates all media devices attached to the system and the media entities
   registered with it, by creating instances of the ``MediaDevice`` class and
   storing them.

-  `DeviceMatch <http://libcamera.org/api-html/classlibcamera_1_1DeviceMatch.html>`_:
   Describes a media device search pattern using entity names, or other
   properties.

-  `V4L2VideoDevice <http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html>`_:
   Models an instance of a V4L2 video device constructed with the path to a V4L2
   video device node.

-  `V4L2SubDevice <http://libcamera.org/api-html/classlibcamera_1_1V4L2Subdevice.html>`_:
   Provides an API to the sub-devices that model the hardware components of a
   V4L2 device.

-  `CameraSensor <http://libcamera.org/api-html/classlibcamera_1_1CameraSensor.html>`_:
   Abstracts camera sensor handling by hiding the details of the V4L2 subdevice
   kernel API and caching sensor information.

-  `CameraData <http://libcamera.org/api-html/classlibcamera_1_1CameraData.html>`_:
   Represents device-specific data a pipeline handler associates to each Camera
   instance.

-  `StreamConfiguration <http://libcamera.org/api-html/structlibcamera_1_1StreamConfiguration.html>`_:
   Models the current configuration of an image stream produced by the camera by
   reporting its format and sizes.

-  `CameraConfiguration <http://libcamera.org/api-html/classlibcamera_1_1CameraConfiguration.html>`_:
   Represents the current configuration of a camera, which includes a list of
   stream configurations for each active stream in a capture session. When
   validated, it is applied to the camera.

-  `IPAInterface <http://libcamera.org/api-html/classlibcamera_1_1IPAInterface.html>`_:
   The interface to the Image Processing Algorithm (IPA) module which performs
   the computation of the image processing pipeline tuning parameters.

-  `ControlList <http://libcamera.org/api-html/classlibcamera_1_1ControlList.html>`_:
   A list of control items, indexed by Control<> instances or by numerical index
   which contains values used by application and IPA to change parameters of
   image streams, used to return to applications and share with IPA the metadata
   associated with the captured images, and to advertise the immutable camera
   characteristics enumerated at system initialization time.

Creating a PipelineHandler
--------------------------

This guide walks through the steps to create a simple pipeline handler
called “Vivid” that supports the `V4L2 Virtual Video Test Driver`_ (vivid).

To use the vivid test driver, you first need to check that the vivid kernel
module is loaded, for example with the ``modprobe vivid`` command.

.. _V4L2 Virtual Video Test Driver: https://www.kernel.org/doc/html/latest/admin-guide/media/vivid.html

Create the skeleton file structure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To add a new pipeline handler, create a directory to hold the pipeline code in
the *src/libcamera/pipeline/* directory that matches the name of the pipeline
(in this case *vivid*). Inside the new directory add a *meson.build* file that
integrates with the libcamera build system, and a *vivid.cpp* file that matches
the name of the pipeline.

In the *meson.build* file, add the *vivid.cpp* file as a build source for
libcamera by adding it to the global meson ``libcamera_sources`` variable:

.. code-block:: none

   # SPDX-License-Identifier: CC0-1.0

   libcamera_sources += files([
       'vivid.cpp',
   ])

Users of libcamera can selectively enable pipelines while building libcamera
using the ``pipelines`` option.

For example, to enable only the IPU3, UVC, and VIVID pipelines, specify them as
a comma separated list with ``-Dpipelines`` when generating a build directory:

.. code-block:: shell

    meson build -Dpipelines=ipu3,uvcvideo,vivid

Read the `Meson build configuration`_ documentation for more information on
configuring a build directory.

.. _Meson build configuration: https://mesonbuild.com/Configuring-a-build-directory.html

To add the new pipeline handler to this list of options, add its directory name
to the libcamera build options in the top level ``meson_options.txt``.

.. code-block:: none

   option('pipelines',
           type : 'array',
           choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc', 'vivid'],
           description : 'Select which pipeline handlers to include')


In *vivid.cpp* add the pipeline handler to the ``libcamera`` namespace, defining
a `PipelineHandler`_ derived class named PipelineHandlerVivid, and add stub
methods for the overridden class members.

.. _PipelineHandler: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html

.. code-block:: cpp

   namespace libcamera {

   class PipelineHandlerVivid : public PipelineHandler
   {
   public:
          PipelineHandlerVivid(CameraManager *manager);

          CameraConfiguration *generateConfiguration(Camera *camera,
          const StreamRoles &roles) override;
          int configure(Camera *camera, CameraConfiguration *config) override;

          int exportFrameBuffers(Camera *camera, Stream *stream,
          std::vector<std::unique_ptr<FrameBuffer>> *buffers) override;

          int start(Camera *camera, const ControlList *controls) override;
          void stop(Camera *camera) override;

          int queueRequestDevice(Camera *camera, Request *request) override;

          bool match(DeviceEnumerator *enumerator) override;
   };

   PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
          : PipelineHandler(manager)
   {
   }

   CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
                                                                    const StreamRoles &roles)
   {
          return nullptr;
   }

   int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
   {
          return -1;
   }

   int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
                                                std::vector<std::unique_ptr<FrameBuffer>> *buffers)
   {
          return -1;
   }

   int PipelineHandlerVivid::start(Camera *camera, const ControlList *controls)
   {
          return -1;
   }

   void PipelineHandlerVivid::stop(Camera *camera)
   {
   }

   int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
   {
          return -1;
   }

   bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
   {
          return false;
   }

   REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid)

   } /* namespace libcamera */

Note that you must register the ``PipelineHandler`` subclass with the pipeline
handler factory using the `REGISTER_PIPELINE_HANDLER`_ macro which
registers it and creates a global symbol to reference the class and make it
available to try and match devices.

.. _REGISTER_PIPELINE_HANDLER: http://libcamera.org/api-html/pipeline__handler_8h.html

For debugging and testing a pipeline handler during development, you can define
a log message category for the pipeline handler. The ``LOG_DEFINE_CATEGORY``
macro and ``LIBCAMERA_LOG_LEVELS`` environment variable help you use the inbuilt
libcamera `logging infrastructure`_ that allow for the inspection of internal
operations in a user-configurable way.

.. _logging infrastructure: http://libcamera.org/api-html/log_8h.html

Add the following before the ``PipelineHandlerVivid`` class declaration:

.. code-block:: cpp

   LOG_DEFINE_CATEGORY(VIVID)

At this point you need the following includes for logging and pipeline handler
features:

.. code-block:: cpp

   #include "libcamera/internal/log.h"
   #include "libcamera/internal/pipeline_handler.h"

Run the following commands:

.. code-block:: shell

   meson build
   ninja -C build

To build the libcamera code base, and confirm that the build system found the
new pipeline handler by running:

.. code-block:: shell

   LIBCAMERA_LOG_LEVELS=Camera:0 ./build/src/cam/cam -l

And you should see output like the below:

.. code-block:: shell

    DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'PipelineHandlerVivid'

Matching devices
~~~~~~~~~~~~~~~~

Each pipeline handler registered in libcamera gets tested against the current
system configuration, by matching a ``DeviceMatch`` with the system
``DeviceEnumerator``. A successful match makes sure all the requested components
have been registered in the system and allows the pipeline handler to be
initialized.

The main entry point of a pipeline handler is the `match()`_ class member
function. When the ``CameraManager`` is started (using the `start()`_ method),
all the registered pipeline handlers are iterated and their ``match`` function
called with an enumerator of all devices it found on a system.

The match method should identify if there are suitable devices available in the
``DeviceEnumerator`` which the pipeline supports, returning ``true`` if it
matches a device, and ``false`` if it does not. To do this, construct a
`DeviceMatch`_ class with the name of the ``MediaController`` device to match.
You can specify the search further by adding specific media entities to the
search using the ``.add()`` method on the DeviceMatch.

.. _match(): https://www.libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a7cd5b652a2414b543ec20ba9dabf61b6
.. _start(): http://libcamera.org/api-html/classlibcamera_1_1CameraManager.html#a49e322880a2a26013bb0076788b298c5
.. _DeviceMatch: http://libcamera.org/api-html/classlibcamera_1_1DeviceMatch.html

This example uses search patterns that match vivid, but when developing a new
pipeline handler, you should change this value to suit your device identifier.

Replace the contents of the ``PipelineHandlerVivid::match`` method with the
following:

.. code-block:: cpp

   DeviceMatch dm("vivid");
   dm.add("vivid-000-vid-cap");
   return false; // Prevent infinite loops for now

With the device matching criteria defined, attempt to acquire exclusive access
to the matching media controller device with the `acquireMediaDevice`_ method.
If the method attempts to acquire a device it has already matched, it returns
``false``.

.. _acquireMediaDevice: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a77e424fe704e7b26094164b9189e0f84

Add the following below ``dm.add("vivid-000-vid-cap");``:

.. code-block:: cpp

   MediaDevice *media = acquireMediaDevice(enumerator, dm);
   if (!media)
           return false;

The pipeline handler now needs an additional include. Add the following to the
existing include block for device enumeration functionality:

.. code-block:: cpp

   #include "libcamera/internal/device_enumerator.h"

At this stage, you should test that the pipeline handler can successfully match
the devices, but have not yet added any code to create a Camera which libcamera
reports to applications.

As a temporary validation step, add a debug print with

.. code-block:: cpp

   LOG(VIVID, Debug) << "Vivid Device Identified";

before the final closing return statement in the ``PipelineHandlerVivid::match``
method for when when the pipeline handler successfully matches the
``MediaDevice`` and ``MediaEntity`` names.

Test that the pipeline handler matches and finds a device by rebuilding, and
running

.. code-block:: shell

   ninja -C build
   LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./build/src/cam/cam -l

And you should see output like the below:

.. code-block:: shell

    DEBUG VIVID vivid.cpp:74 Vivid Device Identified

Creating camera devices
~~~~~~~~~~~~~~~~~~~~~~~

If the pipeline handler successfully matches with the system it is running on,
it can proceed to initialization, by creating all the required instances of the
``V4L2VideoDevice``, ``V4L2Subdevice`` and ``CameraSensor`` hardware abstraction
classes. If the Pipeline handler supports an ISP, it can then also Initialise
the IPA module before proceeding to the creation of the Camera devices.

An image ``Stream`` represents a sequence of images and data of known size and
format, stored in application-accessible memory locations. Typical examples of
streams are the ISP processed outputs and the raw images captured at the
receivers port output.

The Pipeline Handler is responsible for defining the set of Streams associated
with the Camera.

Each Camera has instance-specific data represented using the `CameraData`_
class, which can be extended for the specific needs of the pipeline handler.

.. _CameraData: http://libcamera.org/api-html/classlibcamera_1_1CameraData.html


To support the Camera we will later register, we need to create a CameraData
class that we can implement for our specific Pipeline Handler.

Define a new ``VividCameraData()`` class derived from ``CameraData`` by adding
the following code before the PipelineHandlerVivid class definition where it
will be used:

.. code-block:: cpp

   class VividCameraData : public CameraData
   {
   public:
          VividCameraData(PipelineHandler *pipe, MediaDevice *media)
                : CameraData(pipe), media_(media), video_(nullptr)
          {
          }

          ~VividCameraData()
          {
                delete video_;
          }

          int init();
          void bufferReady(FrameBuffer *buffer);

          MediaDevice *media_;
          V4L2VideoDevice *video_;
          Stream stream_;
   };

This example pipeline handler handles a single video device and supports a
single stream, represented by the ``VividCameraData`` class members. More
complex pipeline handlers might register cameras composed of several video
devices and sub-devices, or multiple streams per camera that represent the
several components of the image capture pipeline. You should represent all these
components in the ``CameraData`` derived class when developing a custom
PipelineHandler.

In our example VividCameraData we implement an ``init()`` function to prepare
the object from our PipelineHandler, however the CameraData class does not
specify the interface for initialisation and PipelineHandlers can manage this
based on their own needs. Derived CameraData classes are used only by their
respective pipeline handlers.

The CameraData class stores the context required for each camera instance and
is usually responsible for opening all Devices used in the capture pipeline.

We can now implement the ``init`` method for our example Pipeline Handler to
create a new V4L2 video device from the media entity, which we can specify using
the `MediaDevice::getEntityByName`_ method from the MediaDevice. As our example
is based upon the simplistic Vivid test device, we only need to open a single
capture device named 'vivid-000-vid-cap' by the device.

.. _MediaDevice::getEntityByName: http://libcamera.org/api-html/classlibcamera_1_1MediaDevice.html#ad5d9279329ef4987ceece2694b33e230

.. code-block:: cpp

   int VividCameraData::init()
   {
          video_ = new V4L2VideoDevice(media_->getEntityByName("vivid-000-vid-cap"));
          if (video_->open())
                return -ENODEV;

          return 0;
   }

The CameraData should be created and initialised before we move on to register a
new Camera device so we need to construct and initialise our
VividCameraData after we have identified our device within
PipelineHandlerVivid::match(). The VividCameraData is wrapped by a
std::unique_ptr to help manage the lifetime of our CameraData instance.

If the camera data initialization fails, return ``false`` to indicate the
failure to the ``match()`` method and prevent retrying of the pipeline handler.

.. code-block:: cpp

   std::unique_ptr<VividCameraData> data = std::make_unique<VividCameraData>(this, media);

   if (data->init())
           return false;


Once the camera data has been initialized, the Camera device instances and the
associated streams have to be registered. Create a set of streams for the
camera, which for this device is only one. You create a camera using the static
`Camera::create`_ method, passing the pipeline handler, the id of the camera,
and the streams available. Then register the camera and its data with the
pipeline handler and camera manager using `registerCamera`_.

Finally with a successful construction, we return 'true' indicating that the
PipelineHandler successfully matched and constructed a device.

.. _Camera::create: http://libcamera.org/api-html/classlibcamera_1_1Camera.html#a453740e0d2a2f495048ae307a85a2574
.. _registerCamera: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#adf02a7f1bbd87aca73c0e8d8e0e6c98b

.. code-block:: cpp

   std::set<Stream *> streams{ &data->stream_ };
   std::shared_ptr<Camera> camera = Camera::create(this, data->video_->deviceName(), streams);
   registerCamera(std::move(camera), std::move(data));

   return true;


Our match function should now look like the following:

.. code-block:: cpp

   bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
   {
   	DeviceMatch dm("vivid");
   	dm.add("vivid-000-vid-cap");

   	MediaDevice *media = acquireMediaDevice(enumerator, dm);
   	if (!media)
   		return false;

   	std::unique_ptr<VividCameraData> data = std::make_unique<VividCameraData>(this, media);

   	/* Locate and open the capture video node. */
   	if (data->init())
   		return false;

   	/* Create and register the camera. */
   	std::set<Stream *> streams{ &data->stream_ };
   	std::shared_ptr<Camera> camera = Camera::create(this, data->video_->deviceName(), streams);
   	registerCamera(std::move(camera), std::move(data));

   	return true;
   }

We will need to use our custom CameraData class frequently throughout the
pipeline handler, so we add a private convenience helper to our Pipeline handler
to obtain and cast the custom CameraData instance from a Camera instance.

.. code-block:: cpp

   private:
       VividCameraData *cameraData(const Camera *camera)
       {
               return static_cast<VividCameraData *>(
                        PipelineHandler::cameraData(camera));
       }

At this point, you need to add the following new includes to provide the Camera
interface, and device interaction interfaces.

.. code-block:: cpp

   #include <libcamera/camera.h>
   #include "libcamera/internal/media_device.h"
   #include "libcamera/internal/v4l2_videodevice.h"

Registering controls and properties
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The libcamera `controls framework`_ allows an application to configure the
streams capture parameters on a per-frame basis and is also used to advertise
immutable properties of the ``Camera`` device.

The libcamera controls and properties are defined in YAML form which is
processed to automatically generate documentation and interfaces. Controls are
defined by the src/libcamera/`control_ids.yaml`_ file and camera properties
are defined by src/libcamera/`properties_ids.yaml`_.

.. _controls framework: http://libcamera.org/api-html/controls_8h.html
.. _control_ids.yaml: http://libcamera.org/api-html/control__ids_8h.html
.. _properties_ids.yaml: http://libcamera.org/api-html/property__ids_8h.html

Pipeline handlers can optionally register the list of controls an application
can set as well as a list of immutable camera properties. Being both
Camera-specific values, they are represented in the ``CameraData`` base class,
which provides two members for this purpose: the `CameraData::controlInfo_`_ and
the `CameraData::properties_`_ fields.

.. _CameraData::controlInfo_: http://libcamera.org/api-html/classlibcamera_1_1CameraData.html#ab9fecd05c655df6084a2233872144a52
.. _CameraData::properties_: http://libcamera.org/api-html/classlibcamera_1_1CameraData.html#a84002c29f45bd35566c172bb65e7ec0b

The ``controlInfo_`` field represents a map of ``ControlId`` instances
associated with the limits of valid values supported for the control. More
information can be found in the `ControlInfoMap`_ class documentation.

.. _ControlInfoMap: http://libcamera.org/api-html/classlibcamera_1_1ControlInfoMap.html

Pipeline handlers register controls to expose the tunable device and IPA
parameters to applications. Our example pipeline handler only exposes trivial
controls of the video device, by registering a ``ControlId`` instance with
associated values for each supported V4L2 control but demonstrates the mapping
of V4L2 Controls to libcamera ControlIDs.

Complete the initialization of the ``VividCameraData`` class by adding the
following code to the ``VividCameraData::init()`` method to initialise the
controls. For more complex control configurations, this could of course be
broken out to a separate function, but for now we just initialise the small set
inline in our CameraData init:

.. code-block:: cpp

   /* Initialise the supported controls. */
   const ControlInfoMap &controls = video_->controls();
   ControlInfoMap::Map ctrls;

   for (const auto &ctrl : controls) {
           const ControlId *id;
           ControlInfo info;

           switch (ctrl.first->id()) {
           case V4L2_CID_BRIGHTNESS:
                   id = &controls::Brightness;
                   info = ControlInfo{ { -1.0f }, { 1.0f }, { 0.0f } };
                   break;
           case V4L2_CID_CONTRAST:
                   id = &controls::Contrast;
                   info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } };
                   break;
           case V4L2_CID_SATURATION:
                   id = &controls::Saturation;
                   info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } };
                   break;
           default:
                   continue;
           }

           ctrls.emplace(id, info);
   }

   controlInfo_ = std::move(ctrls);

The ``properties_`` field is  a list of ``ControlId`` instances
associated with immutable values, which represent static characteristics that can
be used by applications to identify camera devices in the system. Properties can be
registered by inspecting the values of V4L2 controls from the video devices and
camera sensor (for example to retrieve the position and orientation of a camera)
or to express other immutable characteristics. The example pipeline handler does
not register any property, but examples are available in the libcamera code
base.

.. TODO: Add a property example to the pipeline handler. At least the model.

At this point you need to add the following includes to the top of the file for
handling controls:

.. code-block:: cpp

   #include <libcamera/controls.h>
   #include <libcamera/control_ids.h>

Generating a default configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once ``Camera`` devices and the associated ``Streams`` have been registered, an
application can proceed to acquire and configure the camera to prepare it for a
frame capture session.

Applications specify the requested configuration by assigning a
``StreamConfiguration`` instance to each stream they want to enable which
expresses the desired image size and pixel format. The stream configurations are
grouped in a ``CameraConfiguration`` which is inspected by the pipeline handler
and validated to adjust it to a supported configuration. This may involve
adjusting the formats or image sizes or alignments for example to match the
capabilities of the device.

Applications may choose to repeat validation stages, adjusting paramters until a
set of validated StreamConfigurations are returned that is acceptable for the
applications needs. When the pipeline handler receives a valid camera
configuration it can use the image stream configurations to apply settings to
the hardware devices.

This configuration and validation process is managed with another Pipeline
specific class derived from a common base implementation and interface.

To support validation in our example pipeline handler, Create a new class called
``VividCameraConfiguration`` derived from the base `CameraConfiguration`_ class
which we can implement and use within our ``PipelineHandlerVivid`` class.

.. _CameraConfiguration: http://libcamera.org/api-html/classlibcamera_1_1CameraConfiguration.html

The derived ``CameraConfiguration`` class must override the base class
``validate()`` function, where the stream configuration inspection and
adjustment happens.

.. code-block:: cpp

    class VividCameraConfiguration : public CameraConfiguration
    {
    public:
           VividCameraConfiguration();

           Status validate() override;
    };

    VividCameraConfiguration::VividCameraConfiguration()
           : CameraConfiguration()
    {
    }

Applications generate a ``CameraConfiguration`` instance by calling the
`Camera::generateConfiguration()`_ function, which calls into the pipeline
implementation of the overridden `PipelineHandler::generateConfiguration()`_
method.

.. _Camera::generateConfiguration(): http://libcamera.org/api-html/classlibcamera_1_1Camera.html#a25c80eb7fc9b1cf32692ce0c7f09991d
.. _PipelineHandler::generateConfiguration(): http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a7932e87735695500ce1f8c7ae449b65b

Configurations are generated by receiving a list of ``StreamRoles`` instances,
which libcamera uses as predefined ways an application intends to use a camera
(You can read the full list in the `StreamRole API`_ documentation). These are
optional hints on how an application intends to use a stream, and a pipeline
handler should return an ideal configuration for each role that is requested.

.. _StreamRole API: http://libcamera.org/api-html/stream_8h.html#file_a295d1f5e7828d95c0b0aabc0a8baac03

In the pipeline handler ``generateConfiguration`` implementation, remove the
``return nullptr;``, create a new instance of the ``CameraConfiguration``
derived class, and assign it to a base class pointer.

.. code-block:: cpp

   VividCameraData *data = cameraData(camera);
   CameraConfiguration *config = new VividCameraConfiguration();

A ``CameraConfiguration`` is specific to each pipeline, so you can only create
it from the pipeline handler code path. Applications can also generate an empty
configuration and add desired stream configurations manually. Pipelines must
allow for this by returning an empty configuration if no roles are requested.

To support this in our PipelineHandlerVivid, next add the following check in
``generateConfiguration`` after the Cameraconfiguration has been constructed:

.. code-block:: cpp

   if (roles.empty())
           return config;

A production pipeline handler should generate the ``StreamConfiguration`` for
all the appropriate stream roles a camera device supports. For this simpler
example (with only one stream), the pipeline handler always returns the same
configuration, inferred from the underlying V4L2VideoDevice.

How it does this is shown below, but examination of the more full-featured
pipelines for IPU3, RKISP1 and RaspberryPi are recommend to explore more
complex examples.

To generate a ``StreamConfiguration``, you need a list of pixel formats and
frame sizes which supported outputs of the stream. You can fetch a map of the
``V4LPixelFormat`` and ``SizeRange`` supported by the underlying output device,
but the pipeline handler needs to convert this to a ``libcamera::PixelFormat``
type to pass to applications. We do this here using ``std::transform`` to
convert the formats and populate a new ``PixelFormat`` map as shown below.

Continue adding the following code example to our ``generateConfiguration``
implementation.

.. code-block:: cpp

   std::map<V4L2PixelFormat, std::vector<SizeRange>> v4l2Formats =
           data->video_->formats();
   std::map<PixelFormat, std::vector<SizeRange>> deviceFormats;
   std::transform(v4l2Formats.begin(), v4l2Formats.end(),
          std::inserter(deviceFormats, deviceFormats.begin()),
          [&](const decltype(v4l2Formats)::value_type &format) {
              return decltype(deviceFormats)::value_type{
                  format.first.toPixelFormat(),
                  format.second
              };
          });

The `StreamFormats`_ class holds information about the pixel formats and frame
sizes that a stream can support. The class groups size information by the pixel
format, which can produce it.

.. _StreamFormats: http://libcamera.org/api-html/classlibcamera_1_1StreamFormats.html

The code below uses the ``StreamFormats`` class to represent all of the
supported pixel formats, associated with a list of frame sizes. It then
generates a supported StreamConfiguration to model the information an
application can use to configure a single stream.

Continue adding the following code to support this:

.. code-block:: cpp

   StreamFormats formats(deviceFormats);
   StreamConfiguration cfg(formats);

As well as a list of supported StreamFormats, the StreamConfiguration is also
expected to provide an initialsed default configuration. This may be arbitrary,
but depending on use case you may which to select an output that matches the
Sensor output, or prefer a pixelformat which might provide higher performance on
the hardware. The bufferCount represents the number of buffers required to
support functional continuous processing on this stream.

.. code-block:: cpp

   cfg.pixelFormat = formats::BGR888;
   cfg.size = { 1280, 720 };
   cfg.bufferCount = 4;

Finally add each ``StreamConfiguration`` generated to the
``CameraConfiguration``, and ensure that it has been validated before returning
it to the application. With only a single supported stream, this code adds only
a single StreamConfiguration however a StreamConfiguration should be added for
each supported role in a device that can handle more streams.

Add the following code to complete the implementation of
``generateConfiguration``:

.. code-block:: cpp

   config->addConfiguration(cfg);

   config->validate();

   return config;

To validate a camera configuration, a pipeline handler must implement the
`CameraConfiguration::validate()`_ method in it's derived class to inspect all
the stream configuration associated to it, make any adjustments required to make
the configuration valid, and return the validation status.

If changes are made, it marks the configuration as ``Adjusted``, however if the
requested configuration is not supported and cannot be adjusted it shall be
refused and marked as ``Invalid``.

.. _CameraConfiguration::validate(): http://libcamera.org/api-html/classlibcamera_1_1CameraConfiguration.html#a29f8f263384c6149775b6011c7397093

The validation phase makes sure all the platform-specific constraints are
respected by the requested configuration. The most trivial examples being making
sure the requested image formats are supported and the image alignment
restrictions adhered to. The pipeline handler specific implementation of
``validate()`` shall inspect all the configuration parameters received and never
assume they are correct, as applications are free to change the requested stream
parameters after the configuration has been generated.

Again, this example pipeline handler is simpler, look at the more complex
implementations for a realistic example.

Add the following function implementation to your file:

.. code-block:: cpp

   CameraConfiguration::Status VividCameraConfiguration::validate()
   {
           Status status = Valid;

           if (config_.empty())
                  return Invalid;

           if (config_.size() > 1) {
                  config_.resize(1);
                  status = Adjusted;
           }

           StreamConfiguration &cfg = config_[0];

           const std::vector<libcamera::PixelFormat> formats = cfg.formats().pixelformats();
           if (std::find(formats.begin(), formats.end(), cfg.pixelFormat) == formats.end()) {
                  cfg.pixelFormat = cfg.formats().pixelformats()[0];
                  LOG(VIVID, Debug) << "Adjusting format to " << cfg.pixelFormat.toString();
                  status = Adjusted;
           }

           cfg.bufferCount = 4;

           return status;
   }

Now that we are handling the ``PixelFormat`` type, we also need to add
``#include <libcamera/formats.h>`` to the include section before we rebuild the
codebase, and test:

.. code-block:: shell

   ninja -C build
   LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./build/src/cam/cam -c vivid -I

You should see the following output showing the capabilites of our new pipeline
handler, and showing that our configurations have been generated:

.. code-block:: shell

    Using camera vivid
    0: 1280x720-BGR888
    * Pixelformat: NV21 (320x180)-(3840x2160)/(+0,+0)
    - 320x180
    - 640x360
    - 640x480
    - 1280x720
    - 1920x1080
    - 3840x2160
    * Pixelformat: NV12 (320x180)-(3840x2160)/(+0,+0)
    - 320x180
    - 640x360
    - 640x480
    - 1280x720
    - 1920x1080
    - 3840x2160
    * Pixelformat: BGRA8888 (320x180)-(3840x2160)/(+0,+0)
    - 320x180
    - 640x360
    - 640x480
    - 1280x720
    - 1920x1080
    - 3840x2160
    * Pixelformat: RGBA8888 (320x180)-(3840x2160)/(+0,+0)
    - 320x180
    - 640x360
    - 640x480
    - 1280x720
    - 1920x1080
    - 3840x2160

Configuring a device
~~~~~~~~~~~~~~~~~~~~

With the configuration generated, and optionally modified and re-validated, a
pipeline handler needs a method that allows an application to apply a
configuration to the hardware devices.

The `PipelineHandler::configure()`_ method receives a valid
`CameraConfiguration`_ and applies the settings to hardware devices, using its
parameters to prepare a device for a streaming session with the desired
properties.

.. _PipelineHandler::configure(): http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a930f2a9cdfb51dfb4b9ca3824e84fc29
.. _CameraConfiguration: http://libcamera.org/api-html/classlibcamera_1_1CameraConfiguration.html

Replace the contents of the stubbed ``PipelineHandlerVivid::configure`` method
with the following to obtain the camera data and stream configuration. This
pipeline handler supports only a single stream, so it directly obtains the first
``StreamConfiguration`` from the camera configuration. A pipeline handler with
multiple streams should inspect each StreamConfiguration and configure the
system accordingly.

.. code-block:: cpp

   VividCameraData *data = cameraData(camera);
   StreamConfiguration &cfg = config->at(0);
   int ret;

The Vivid capture device is a V4L2 video device, so we use a `V4L2DeviceFormat`_
with the fourcc and size attributes to apply directly to the capture device
node. The fourcc attribute is a `V4L2PixelFormat`_ and differs from the
``libcamera::PixelFormat``. Converting the format requires knowledge of the
plane configuration for multiplanar formats, so you must explicitly convert it
using the helper ``V4L2VideoDevice::toV4L2PixelFormat()`` provided by the
V4L2VideoDevice instance of which the format will be applied on.

.. _V4L2DeviceFormat: http://libcamera.org/api-html/classlibcamera_1_1V4L2DeviceFormat.html
.. _V4L2PixelFormat: http://libcamera.org/api-html/classlibcamera_1_1V4L2PixelFormat.html

Add the following code beneath the code from above:

.. code-block:: cpp

   V4L2DeviceFormat format = {};
   format.fourcc = data->video_->toV4L2PixelFormat(cfg.pixelFormat);
   format.size = cfg.size;

Set the video device format defined above using the
`V4L2VideoDevice::setFormat()`_ method. You should check if the kernel
driver has adjusted the format, as this shows the pipeline handler has failed to
handle the validation stages correctly, and the configure operation shall also
fail.

.. _V4L2VideoDevice::setFormat(): http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#ad67b47dd9327ce5df43350b80c083cca

Continue the implementation with the following code:

.. code-block:: cpp

   ret = data->video_->setFormat(&format);
   if (ret)
          return ret;

   if (format.size != cfg.size ||
          format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat))
          return -EINVAL;

Finally, store and set stream-specific data reflecting the state of the stream.
Associate the configuration with the stream by using the
`StreamConfiguration::setStream`_ method, and set the values of individual
stream configuration members as required.

.. _StreamConfiguration::setStream: http://libcamera.org/api-html/structlibcamera_1_1StreamConfiguration.html#a74a0eb44dad1b00112c7c0443ae54a12

.. NOTE: the cfg.setStream() call here associates the stream to the
   StreamConfiguration however that should quite likely be done as part of
   the validation process. TBD

Complete the configure implementation with the following code:

.. code-block:: cpp

   cfg.setStream(&data->stream_);
   cfg.stride = format.planes[0].bpl;

   return 0;

.. TODO: stride SHALL be assigned in validate

Initializing device controls
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Pipeline handlers can optionally initialize the video devices and camera sensor
controls at system configuration time, to make sure they are defaulted to sane
values. Handling of device controls is again performed using the libcamera
`controls framework`_.

.. _Controls Framework: http://libcamera.org/api-html/controls_8h.html

This section is particularly specific to Vivid as it sets the initial values of
controls to match `Vivid Controls`_ defined by the kernel driver. You won’t need
any of the code below for your pipeline handler, but it’s included as an example
of how to implement functionality your pipeline handler might need.

.. _Vivid Controls: https://www.kernel.org/doc/html/latest/admin-guide/media/vivid.html#controls

We need to add some definitions at the top of the file for convenience. These
come directly from the kernel sources:

.. code-block:: cpp

   #define VIVID_CID_VIVID_BASE            (0x00f00000 | 0xf000)
   #define VIVID_CID_VIVID_CLASS           (0x00f00000 | 1)
   #define VIVID_CID_TEST_PATTERN          (VIVID_CID_VIVID_BASE  + 0)
   #define VIVID_CID_OSD_TEXT_MODE         (VIVID_CID_VIVID_BASE  + 1)
   #define VIVID_CID_HOR_MOVEMENT          (VIVID_CID_VIVID_BASE  + 2)

We can now use the V4L2 control IDs to prepare a list of controls with the
`ControlList`_ class, and set them using the `ControlList::set()`_ method.

.. _ControlList: http://libcamera.org/api-html/classlibcamera_1_1ControlList.html
.. _ControlList::set(): http://libcamera.org/api-html/classlibcamera_1_1ControlList.html#a74a1a29abff5243e6e37ace8e24eb4ba

In our pipeline ``configure`` method, add the following code after the format
has been set and checked to initialise the ControlList and apply it to the
device:

.. code-block:: cpp

   ControlList controls(data->video_->controls());
   controls.set(VIVID_CID_TEST_PATTERN, 0);
   controls.set(VIVID_CID_OSD_TEXT_MODE, 0);

   controls.set(V4L2_CID_BRIGHTNESS, 128);
   controls.set(V4L2_CID_CONTRAST, 128);
   controls.set(V4L2_CID_SATURATION, 128);

   controls.set(VIVID_CID_HOR_MOVEMENT, 5);

   ret = data->video_->setControls(&controls);
   if (ret) {
          LOG(VIVID, Error) << "Failed to set controls: " << ret;
          return ret < 0 ? ret : -EINVAL;
   }

These controls configure VIVID to use a default test pattern, and enable all
on-screen display text, while configuring sensible brightness, contrast and
saturation values. Use the ``controls.set`` method to set individual controls.

Buffer handling and stream control
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once the system has been configured with the requested parameters, it is now
possible for applications to start capturing frames from the ``Camera`` device.

libcamera implements a per-frame request capture model, realized by queueing
``Request`` instances to a ``Camera`` object. Before applications can start
submitting capture requests the capture pipeline needs to be prepared to deliver
frames as soon as they are requested. Memory should be initialized and made
available to the devices which have to be started and ready to produce
images. At the end of a capture session the ``Camera`` device needs to be
stopped, to gracefully clean up any allocated memory and stop the hardware
devices. Pipeline handlers implement two methods for these purposes, the
``start()`` and ``stop()`` methods.

The memory initialization phase that happens at ``start()`` time serves to
configure video devices to be able to use memory buffers exported as dma-buf
file descriptors. From the pipeline handlers perspective the video devices that
provide application facing streams always act as memory importers which use,
in V4L2 terminology, buffers of V4L2_MEMORY_DMABUF memory type.

libcamera also provides an API to allocate and export memory to applications
realized through the `exportFrameBuffers`_ function and the
`FrameBufferAllocator`_ class which will be presented later.

.. _exportFrameBuffers: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a6312a69da7129c2ed41f9d9f790adf7c
.. _FrameBufferAllocator: http://libcamera.org/api-html/classlibcamera_1_1FrameBufferAllocator.html

Please refer to the V4L2VideoDevice API documentation, specifically the
`allocateBuffers`_, `importBuffers`_ and `exportBuffers`_ functions for a
detailed description of the video device memory management.

.. _allocateBuffers: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#a3a1a77e5e6c220ea7878e89485864a1c
.. _importBuffers: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#a154f5283d16ebd5e15d63e212745cb64
.. _exportBuffers: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#ae9c0b0a68f350725b63b73a6da5a2ecd

Video memory buffers are represented in libcamera by the `FrameBuffer`_ class.
A ``FrameBuffer`` instance has to be associated to each ``Stream`` which is part
of a capture ``Request``. Pipeline handlers should prepare the capture devices
by importing the dma-buf file descriptors it needs to operate on. This operation
is performed by using the ``V4L2VideoDevice`` API, which provides an
``importBuffers()`` function that prepares the video device accordingly.

.. _FrameBuffer: http://libcamera.org/api-html/classlibcamera_1_1FrameBuffer.html

Implement the pipeline handler ``start()`` function by replacing the stub
version with the following code:

.. code-block:: c++

   VividCameraData *data = cameraData(camera);
   unsigned int count = data->stream_.configuration().bufferCount;

   int ret = data->video_->importBuffers(count);
   if (ret < 0)
         return ret;

   return 0;

During the startup phase pipeline handlers allocate any internal buffer pool
required to transfer data between different components of the image capture
pipeline, for example, between the CSI-2 receiver and the ISP input. The example
pipeline does not require any internal pool, but examples are available in more
complex pipeline handlers in the libcamera code base.

Applications might want to use memory allocated in the video devices instead of
allocating it from other parts of the system. libcamera provides an abstraction
to assist with this task in the `FrameBufferAllocator`_ class. The
``FrameBufferAllocator`` reserves memory for a ``Stream`` in the video device
and exports it as dma-buf file descriptors. From this point on, the allocated
``FrameBuffer`` are associated to ``Stream`` instances in a ``Request`` and then
imported by the pipeline hander in exactly the same fashion as if they were
allocated elsewhere.

.. _FrameBufferAllocator: http://libcamera.org/api-html/classlibcamera_1_1FrameBufferAllocator.html

Pipeline handlers support the ``FrameBufferAllocator`` operations by
implementing the `exportFrameBuffers`_ function, which will allocate memory in
the video device associated with a stream and export it.

.. _exportFrameBuffers: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a6312a69da7129c2ed41f9d9f790adf7c

Implement the ``exportFrameBuffers`` stub method with the following code to
handle this:

.. code-block:: cpp

   unsigned int count = stream->configuration().bufferCount;
   VividCameraData *data = cameraData(camera);

   return data->video_->exportBuffers(count, buffers);

Once memory has been properly setup, the video devices can be started, to
prepare for capture operations. Complete the ``start`` method implementation
with the following code:

.. code-block:: cpp

   ret = data->video_->streamOn();
   if (ret < 0) {
          data->video_->releaseBuffers();
          return ret;
   }

   return 0;

The method starts the video device associated with the stream with the
`streamOn`_ method. If the call fails, the error value is propagated to the
caller and the `releaseBuffers`_ method releases any buffers to leave the device
in a consistent state. If your pipeline handler uses any image processing
algorithms, or other devices you should also stop them.

.. _streamOn: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#a588a5dc9d6f4c54c61136ac43ff9a8cc
.. _releaseBuffers: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#a191619c152f764e03bc461611f3fcd35

Of course we also need to handle the corresponding actions to stop streaming on
a device, Add the following to the ``stop`` method, to stop the stream with the
`streamOff`_ method and release all buffers.

.. _streamOff: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#a61998710615bdf7aa25a046c8565ed66

.. code-block:: cpp

   VividCameraData *data = cameraData(camera);
   data->video_->streamOff();
   data->video_->releaseBuffers();

Queuing requests between applications and hardware
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

libcamera implements a streaming model based on capture requests queued by an
application to the ``Camera`` device. Each request contains at least one
``Stream`` instance with an associated ``FrameBuffer`` object.

When an application sends a capture request, the pipeline handler identifies
which video devices have to be provided with buffers to generate a frame from
the enabled streams.

This example pipeline handler identifies the buffer using the `findBuffer`_
helper from the only supported stream and queues it to the capture device
directly with the `queueBuffer`_ method provided by the V4L2VideoDevice.

.. _findBuffer: http://libcamera.org/api-html/classlibcamera_1_1Request.html#ac66050aeb9b92c64218945158559c4d4
.. _queueBuffer: http://libcamera.org/api-html/classlibcamera_1_1V4L2VideoDevice.html#a594cd594686a8c1cf9ae8dba0b2a8a75

Replace the stubbed contents of ``queueRequestDevice`` with the following:

.. code-block:: cpp

   VividCameraData *data = cameraData(camera);
   FrameBuffer *buffer = request->findBuffer(&data->stream_);
   if (!buffer) {
          LOG(VIVID, Error)
                  << "Attempt to queue request with invalid stream";

          return -ENOENT;
   }

   int ret = data->video_->queueBuffer(buffer);
   if (ret < 0)
          return ret;

   return 0;

Processing controls
~~~~~~~~~~~~~~~~~~~

Capture requests not only contain streams and memory buffers, but can
optionally contain a list of controls the application has set to modify the
streaming parameters.

Applications can set controls registered by the pipeline handler in the
initialization phase, as explained in the `Registering controls and properties`_
section.

Implement a ``processControls`` method above the ``queueRequestDevice`` method
to loop through the control list received with each request, and inspect the
control values. Controls may need to be converted between the libcamera control
range definitions and their corresponding values on the device before being set.

.. code-block:: cpp

   int PipelineHandlerVivid::processControls(VividCameraData *data, Request *request)
   {
          ControlList controls(data->video_->controls());

          for (auto it : request->controls()) {
                 unsigned int id = it.first;
                 unsigned int offset;
                 uint32_t cid;

                 if (id == controls::Brightness) {
                        cid = V4L2_CID_BRIGHTNESS;
                        offset = 128;
                 } else if (id == controls::Contrast) {
                        cid = V4L2_CID_CONTRAST;
                        offset = 0;
                 } else if (id == controls::Saturation) {
                        cid = V4L2_CID_SATURATION;
                        offset = 0;
                 } else {
                        continue;
                 }

                 int32_t value = lroundf(it.second.get<float>() * 128 + offset);
                 controls.set(cid, std::clamp(value, 0, 255));
          }

          for (const auto &ctrl : controls)
                 LOG(VIVID, Debug)
                        << "Setting control " << utils::hex(ctrl.first)
                        << " to " << ctrl.second.toString();

          int ret = data->video_->setControls(&controls);
          if (ret) {
                 LOG(VIVID, Error) << "Failed to set controls: " << ret;
                 return ret < 0 ? ret : -EINVAL;
          }

          return ret;
   }

Declare the function prototype for the ``processControls`` method within the
private ``PipelineHandlerVivid`` class members, as it is only used internally as
a helper when processing Requests.

.. code-block:: cpp

   private:
        int processControls(VividCameraData *data, Request *request);

A pipeline handler is responsible for applying controls provided in a Request to
the relevant hardware devices. This could be directly on the capture device, or
where appropriate by setting controls on V4L2Subdevices directly. Each pipeline
handler is responsible for understanding the correct procedure for applying
controls to the device they support.

This example pipeline handler applies controls during the `queueRequestDevice`_
method for each request, and applies them to the capture device through the
capture node.

.. _queueRequestDevice: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html#a106914cca210640c9da9ee1f0419e83c

In the ``queueRequestDevice`` method, replace the following:

.. code-block:: cpp

   int ret = data->video_->queueBuffer(buffer);
   if (ret < 0)
        return ret;

With the following code:

.. code-block:: cpp

   int ret = processControls(data, request);
   if (ret < 0)
        return ret;

   ret = data->video_->queueBuffer(buffer);
   if (ret < 0)
        return ret;

We also need to add the following include directive to support the control
value translation operations:

.. code-block:: cpp

   #include <math.h>

Frame completion and event handling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

libcamera implements a signals and slots mechanism (similar to `Qt Signals and
Slots`_) to connect event sources with callbacks to handle them.

As a general summary, a ``Slot`` can be connected to a ``Signal``, which when
emitted triggers the execution of the connected slots.  A detailed description
of the libcamera implementation is available in the `libcamera Signal and Slot`_
classes documentation.

.. _Qt Signals and Slots: https://doc.qt.io/qt-5/signalsandslots.html
.. _libcamera Signal and Slot: http://libcamera.org/api-html/classlibcamera_1_1Signal.html#details

In order to notify applications about the availability of new frames and data,
the ``Camera`` device exposes two ``Signals`` which applications can connect to
be notified of frame completion events. The ``bufferComplete`` signal serves to
report to applications the completion event of a single ``Stream`` part of a
``Request``, while the ``requestComplete`` signal notifies the completion of all
the ``Streams`` and data submitted as part of a request. This mechanism allows
implementation of partial request completion, which allows an application to
inspect completed buffers associated with the single streams without waiting for
all of them to be ready.

The ``bufferComplete`` and ``requestComplete`` signals are emitted by the
``Camera`` device upon notifications received from the pipeline handler, which
tracks the buffers and request completion status.

The single buffer completion notification is implemented by pipeline handlers by
`connecting`_ the ``bufferReady`` signal of the capture devices they have queued
buffers to, to a member function slot that handles processing of the completed
frames. When a buffer is ready, the pipeline handler must propagate the
completion of that buffer to the Camera by using the PipelineHandler base class
``completeBuffer`` function. When all of the buffers referenced by a ``Request``
have been completed, the pipeline handler must again notify the ``Camera`` using
the PipelineHandler base class ``completeRequest`` function. The PipelineHandler
class implementation makes sure the request completion notifications are
delivered to applications in the same order as they have been submitted.

.. _connecting: http://libcamera.org/api-html/classlibcamera_1_1Signal.html#aa04db72d5b3091ffbb4920565aeed382

Returning to the ``int VividCameraData::init()`` method, add the following above
the closing ``return 0;`` to connect the pipeline handler ``bufferReady``
method to the V4L2 device buffer signal.

.. code-block:: cpp

   video_->bufferReady.connect(this, &VividCameraData::bufferReady);

Create the matching ``VividCameraData::bufferReady`` method after your
VividCameradata::init() impelementation.

The ``bufferReady`` method obtains the request from the buffer using the
``request`` method, and notifies the ``Camera`` that the buffer and
request are completed. In this simpler pipeline handler, there is only one
stream, so it completes the request immediately. You can find a more complex
example of event handling with supporting multiple streams in the libcamera
code-base.

.. TODO: Add link

.. code-block:: cpp

   void VividCameraData::bufferReady(FrameBuffer *buffer)
   {
          Request *request = buffer->request();

          pipe_->completeBuffer(request, buffer);
          pipe_->completeRequest(request);
   }

Testing a pipeline handler
~~~~~~~~~~~~~~~~~~~~~~~~~~

Once you've built the pipeline handler, we can rebuild the code base, and test
capture through the pipeline through both of the cam and qcam utilities.

.. code-block:: shell

   ninja -C build
   ./build/src/cam/cam -c vivid -C5

To test that the pipeline handler can detect a device, and capture input.

Running the command above outputs (a lot of) information about pixel formats,
and then starts capturing frame data, and should provide an output such as the
following:

.. code-block:: none

   user@dev:/home/libcamera$ ./build/src/cam/cam -c vivid -C5
   [42:34:08.573066847] [186470]  INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/libcamera/build/src/ipa' to the IPA search path
   [42:34:08.575908115] [186470]  INFO Camera camera_manager.cpp:287 libcamera v0.0.11+876-7b27d262
   [42:34:08.610334268] [186471]  INFO IPAProxy ipa_proxy.cpp:122 libcamera is not installed. Loading IPA configuration from '/home/libcamera/src/ipa/vimc/data'
   Using camera vivid
   [42:34:08.618462130] [186470]  WARN V4L2 v4l2_pixelformat.cpp:176 Unsupported V4L2 pixel format Y10
   ... <remaining Unsupported V4L2 pixel format warnings can be ignored>
   [42:34:08.619901297] [186470]  INFO Camera camera.cpp:793 configuring streams: (0) 1280x720-BGR888
   Capture 5 frames
   fps: 0.00 stream0 seq: 000000 bytesused: 2764800
   fps: 4.98 stream0 seq: 000001 bytesused: 2764800
   fps: 5.00 stream0 seq: 000002 bytesused: 2764800
   fps: 5.03 stream0 seq: 000003 bytesused: 2764800
   fps: 5.03 stream0 seq: 000004 bytesused: 2764800

This demonstrates that the pipeline handler is successfully capturing frames,
but it is helpful to see the visual output and validate the images are being
processed correctly. The libcamera project also implements a Qt based
application which will render the frames in a window for visual inspection:

.. code-block:: shell

   ./build/src/qcam/qcam -c vivid

.. TODO: Running qcam with the vivid pipeline handler appears to have a bug and
         no visual frames are seen. However disabling zero-copy on qcam renders
         them successfully.
generated by cgit v1.2.1 (git 2.18.0) at 2024-10-04 10:09:48 +0000