v4l2: v4l2_compat: Intercept open64, openat64, and mmap64

Some applications (eg. Firefox, Google Chrome, Skype) use open64,
openat64, and mmap64 instead of their non-64 versions that we currently
intercept. Intercept these calls as well. _LARGEFILE64_SOURCE needs to
be set so that the 64-bit symbols are available and not synonymous to
the non-64-bit versions on 64-bit systems.

Also, since we set _FILE_OFFSET_BITS to 32 to force the various open and
mmap symbols that we export to not be the 64-bit versions, our dlsym to
get the original open and mmap calls will not automatically be converted
to their 64-bit versions. Since we intercept both 32-bit and 64-bit
versions of open and mmap, we should be using the 64-bit version to
service both. Fetch the 64-bit versions of openat and mmap directly.

musl defines the 64-bit symbols as macros that are equivalent to the
non-64-bit symbols, so we put compile guards that check if the 64-bit
symbols are defined.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> # Compile with musl
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

1 files changed, 3 insertions, 2 deletions

diff --git a/src/v4l2/v4l2_compat_manager.h b/src/v4l2/v4l2_compat_manager.h
index 14338a59..3d4e512e 100644
--- a/src/v4l2/v4l2_compat_manager.h
+++ b/src/v4l2/v4l2_compat_manager.h
@@ -11,6 +11,7 @@
 #include <fcntl.h>
 #include <map>
 #include <memory>
+#include <sys/mman.h>
 #include <sys/types.h>
 #include <vector>
 
@@ -30,7 +31,7 @@ public:
 		using close_func_t = int (*)(int fd);
 		using ioctl_func_t = int (*)(int fd, unsigned long request, ...);
 		using mmap_func_t = void *(*)(void *addr, size_t length, int prot,
-					      int flags, int fd, off_t offset);
+					      int flags, int fd, off64_t offset);
 		using munmap_func_t = int (*)(void *addr, size_t length);
 
 		openat_func_t openat;
@@ -51,7 +52,7 @@ public:
 	int dup(int oldfd);
 	int close(int fd);
 	void *mmap(void *addr, size_t length, int prot, int flags,
-		   int fd, off_t offset);
+		   int fd, off64_t offset);
 	int munmap(void *addr, size_t length);
 	int ioctl(int fd, unsigned long request, void *arg);