pivid

Experimental video code for Linux / Raspberry Pi

Building

1. Run ./dev_setup.py. (Works On My Machine™, YMMV)
2. Run ninja -C build to run the actual build (this is the only part to repeat after edits).
3. Run binaries from build (like build/pivid_test_decode).
4. If things get weird, rm -rf build and start over with dev_setup.py.

Notable programs:

• pivid_test_decode - play an H.264 video file via KMS/DRM (stop X first)
• pivid_inspect_avformat - print a summary of streams in a video file
• pivid_inspect_kms - print capabilities & properties of KMS/DRM devices
• pivid_inspect_v4l2 - print capabilities & properties of V4L2 devices

Use --help to see usage (and/or read the source).

Notes and links

Raspberry Pi specifics

• All about accelerated video on the Raspberry Pi - my notes
• kernel.org: V3D Graphics Driver - RPi 4 GPU kernel driver docs
• rpi kernel source: drivers/gpu/drm/v3d - RPi 4 GPU kernel driver source
• rpi kernel source: include/uapi/v3d_drm.h - ioctl defs for RPi 4 GPU kernel driver

Graphics output: DRM and KMS

• Wikipedia: Direct Rendering Manager - a good overview
• Blog post: "From pre-history to beyond the global thermonuclear war" - Linux graphics history
• LWN article: Atomic mode setting design overview - the current KMS API
• Man page: Direct Rendering Manager - Kernel Mode-Setting - incomplete but helpful
• Man page: Direct Rendering Manager - Memory Management - incomplete but helpful
• kernel.org: Linux GPU Driver Userland Interfaces - basic notes on the kernel/user interface
• kernel.org: KMS Properties - exhaustive object property list
• kernel source: include/uapi/drm/drm.h and drm_mode.h - kernel/user headers
• ST Micro: DRM/KMS Overview - decent general docs from a chip vendor
• ST Micro: How to trace and debug the framework - an especially useful section
• NVIDIA Jetson Linux API: Direct Rendering Manager - API reference, a bit NVIDIA-specific
• libdrm - library wrapper; see xf86drm.h and xf86drmMode.h (not X-specific despite "xf86")
• libgbm - GPU allocation helper library; see gbm.h
• modetest - command line tool (in the libdrm-tests Debian package)
• kmscube - oft-referenced KMS/GL example program
• kms++ - C++ KMS wrapper & utilities

Notes on DRM and KMS:

• These interfaces are almost completely undocumented. Learn by examples.
• But, many examples use "legacy" interfaces, prefer "atomic" update interfaces.

Video decoding: V4L2

• kernel.org: Video for Linux API - kernel/user interface
• kernel.org: Video for Linux - Memory-to-Memory Stateful Video Decoder Interface
• kernel source: include/uapi/linux/videodev2.h - kernel/user header
• libv4l - thin library wrapper with format conversion; see libv4l2.h
• v4l-utils - useful tools, especially v4l2-ctl
• libavformat - for unpacking containers (.mp4, .mkv); see avformat.h and avio.h

Zero-copy buffer sharing

• kernel.org: Buffer Sharing and Synchronization - kernel buffer management (and user interface)
• kernel.org: Linux GPU Memory Management - PRIME buffer sharing - exporting GPU buffers as "dma-buf" objects
• kernel.org: Video for Linux - Streaming I/O (DMA buffer importing) - using "dma-buf" objects in V4L2
• v4l2test - test/example of V4L2 H.264 decoding feeding KMS

Notes on memory management:

• GPUs have all kinds of memory architectures. The RPi is simple, everything is in system RAM (no dedicated GPU RAM).
• DRM/KMS requires you to create a "framebuffer" object, giving it a "buffer object" ("BO") handle (opaque int32).
• How a "buffer object" is allocated and managed is dependent on the kernel GPU driver.
• Most drivers use the "Graphics Execution Manager" (GEM), their buffer object handles are "GEM handles".
• Most drivers support "dumb buffers", allowing simple creation of a buffer in system RAM.
• Many drivers have more elaborate driver-specific interfaces for allocating various kinds of memory.
• The libgbm ("Generic Buffer Manager") library tries to abstract over those driver-specific interfaces.
• "DRM-PRIME" is a fancy name for ioctl's (see libdrm drmPrimeHandleToFD and drmPrimeFDToHandle) to convert GEM handles to/from "dma-buf" descriptors.
• These "dma-buf" descriptors can be used with other systems like V4L2, of course you have to get the data format right.