Buffers
Prev | Chapter 3. Input/Output | Next

## Buffers

A buffer contains data exchanged by application and driver using one of the Streaming I/O methods. In the multi-planar API, the data is held in planes, while the buffer structure acts as a container for the planes. Only pointers to buffers (planes) are exchanged, the data itself is not copied. These pointers, together with meta-information like timestamps or field parity, are stored in a struct v4l2_buffer, argument to the `VIDIOC_QUERYBUF`, `VIDIOC_QBUF` and `VIDIOC_DQBUF` ioctl. In the multi-planar API, some plane-specific members of struct v4l2_buffer, such as pointers and sizes for each plane, are stored in struct v4l2_plane instead. In that case, struct v4l2_buffer contains an array of plane structures.

Nominally timestamps refer to the first data byte transmitted. In practice however the wide range of hardware covered by the V4L2 API limits timestamp accuracy. Often an interrupt routine will sample the system clock shortly after the field or frame was stored completely in memory. So applications must expect a constant difference up to one field or frame period plus a small (few scan lines) random error. The delay and error can be much larger due to compression or transmission over an external bus when the frames are not properly stamped by the sender. This is frequently the case with USB cameras. Here timestamps refer to the instant the field or frame was received by the driver, not the capture time. These devices identify by not enumerating any video standards, see the section called “Video Standards”.

Similar limitations apply to output timestamps. Typically the video hardware locks to a clock controlling the video timing, the horizontal and vertical synchronization pulses. At some point in the line sequence, possibly the vertical blanking, an interrupt routine samples the system clock, compares against the timestamp and programs the hardware to repeat the previous field or frame, or to display the buffer contents.

Apart of limitations of the video device and natural inaccuracies of all clocks, it should be noted system time itself is not perfectly stable. It can be affected by power saving cycles, warped to insert leap seconds, or even turned back or forth by the system administrator affecting long term measurements. [19]

Table 3.1. struct v4l2_buffer

__u32 | `index` |  | Number of the buffer, set by the application. This field is only used for memory mapping I/O and can range from zero to the number of buffers allocated with the `VIDIOC_REQBUFS` ioctl (struct v4l2_requestbuffers `count`) minus one.
__u32 | `type` |  | Type of the buffer, same as struct v4l2_format `type` or struct v4l2_requestbuffers `type`, set by the application. See Table 3.3, “enum v4l2_buf_type”
__u32 | `bytesused` |  | The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when `type` refers to an input stream, applications when an output stream.
__u32 | `flags` |  | Flags set by the application or driver, see Table 3.4, “Buffer Flags”.
__u32 | `field` |  | Indicates the field order of the image in the buffer, see Table 3.9, “enum v4l2_field”. This field is not used when the buffer contains VBI data. Drivers must set it when `type` refers to an input stream, applications when an output stream.

struct timeval

`timestamp`

For input streams this is time when the first data byte was captured, as returned by the `clock_gettime()` function for the relevant clock id; see `V4L2_BUF_FLAG_TIMESTAMP_*` in Table 3.4, “Buffer Flags”. For output streams the data will not be displayed before this time, secondary to the nominal frame rate determined by the current video standard in enqueued order. Applications can for example zero this field to display frames as soon as possible. The driver stores the time at which the first data byte was actually sent out in the `timestamp` field. This permits applications to monitor the drift between the video and system clock.

struct v4l2_timecode

`timecode`

When `type` is `V4L2_BUF_TYPE_VIDEO_CAPTURE` and the `V4L2_BUF_FLAG_TIMECODE` flag is set in `flags`, this structure contains a frame timecode. In V4L2_FIELD_ALTERNATE mode the top and bottom field contain the same timecode. Timecodes are intended to help video editing and are typically recorded on video tapes, but also embedded in compressed formats like MPEG. This field is independent of the `timestamp` and `sequence` fields.

__u32 | `sequence` |  | Set by the driver, counting the frames (not fields!) in sequence. This field is set for both input and output devices.

In V4L2_FIELD_ALTERNATE mode the top and bottom field have the same sequence number. The count starts at zero and includes dropped or repeated frames. A dropped frame was received by an input device but could not be stored due to lack of free buffer space. A repeated frame was displayed again by an output device because the application did not pass new data in time.

Note this may count the frames received e.g. over USB, without taking into account the frames dropped by the remote hardware due to limited compression throughput or bus bandwidth. These devices identify by not enumerating any video standards, see the section called “Video Standards”.

__u32 | `memory` |  | This field must be set by applications and/or drivers in accordance with the selected I/O method. See Table 3.5, “enum v4l2_memory”
union | `m` |  |
| __u32 | `offset` | For the single-planar API and when `memory` is `V4L2_MEMORY_MMAP` this is the offset of the buffer from the start of the device memory. The value is returned by the driver and apart of serving as parameter to the `mmap()` function not useful for applications. See the section called “Streaming I/O (Memory Mapping)” for details
| unsigned long | `userptr` | For the single-planar API and when `memory` is `V4L2_MEMORY_USERPTR` this is a pointer to the buffer (casted to unsigned long type) in virtual memory, set by the application. See the section called “Streaming I/O (User Pointers)” for details.
| struct v4l2_plane | `*planes` | When using the multi-planar API, contains a userspace pointer to an array of struct v4l2_plane. The size of the array should be put in the `length` field of this v4l2_buffer structure.
| int | `fd` | For the single-plane API and when `memory` is `V4L2_MEMORY_DMABUF` this is the file descriptor associated with a DMABUF buffer.
__u32 | `length` |  | Size of the buffer (not the payload) in bytes for the single-planar API. For the multi-planar API the application sets this to the number of elements in the `planes` array. The driver will fill in the actual number of valid elements in that array.
__u32 | `reserved2` |  | A place holder for future extensions. Applications should set this to 0.
__u32 | `reserved` |  | A place holder for future extensions. Applications should set this to 0.

Table 3.2. struct v4l2_plane

__u32 | `bytesused` |  | The number of bytes occupied by data in the plane (its payload).
__u32 | `length` |  | Size in bytes of the plane (not its payload).
union | `m` |  |
| __u32 | `mem_offset` | When the memory type in the containing struct v4l2_buffer is `V4L2_MEMORY_MMAP`, this is the value that should be passed to `mmap()`, similar to the `offset` field in struct v4l2_buffer.
| unsigned long | `userptr` | When the memory type in the containing struct v4l2_buffer is `V4L2_MEMORY_USERPTR`, this is a userspace pointer to the memory allocated for this plane by an application.
| int | `fd` | When the memory type in the containing struct v4l2_buffer is `V4L2_MEMORY_DMABUF`, this is a file descriptor associated with a DMABUF buffer, similar to the `fd` field in struct v4l2_buffer.
__u32 | `data_offset` |  | Offset in bytes to video data in the plane, if applicable.
__u32 | `reserved[11]` |  | Reserved for future use. Should be zeroed by an application.

Table 3.3. enum v4l2_buf_type

`V4L2_BUF_TYPE_VIDEO_CAPTURE` | 1 | Buffer of a single-planar video capture stream, see the section called “Video Capture Interface”.
`V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE` | 9 | Buffer of a multi-planar video capture stream, see the section called “Video Capture Interface”.
`V4L2_BUF_TYPE_VIDEO_OUTPUT` | 2 | Buffer of a single-planar video output stream, see the section called “Video Output Interface”.
`V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE` | 10 | Buffer of a multi-planar video output stream, see the section called “Video Output Interface”.
`V4L2_BUF_TYPE_VIDEO_OVERLAY` | 3 | Buffer for video overlay, see the section called “Video Overlay Interface”.
`V4L2_BUF_TYPE_VBI_CAPTURE` | 4 | Buffer of a raw VBI capture stream, see the section called “Raw VBI Data Interface”.
`V4L2_BUF_TYPE_VBI_OUTPUT` | 5 | Buffer of a raw VBI output stream, see the section called “Raw VBI Data Interface”.
`V4L2_BUF_TYPE_SLICED_VBI_CAPTURE` | 6 | Buffer of a sliced VBI capture stream, see the section called “Sliced VBI Data Interface”.
`V4L2_BUF_TYPE_SLICED_VBI_OUTPUT` | 7 | Buffer of a sliced VBI output stream, see the section called “Sliced VBI Data Interface”.
`V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY` | 8 | Buffer for video output overlay (OSD), see the section called “Video Output Overlay Interface”.

Table 3.4. Buffer Flags

`V4L2_BUF_FLAG_MAPPED` | 0x0001 | The buffer resides in device memory and has been mapped into the application's address space, see the section called “Streaming I/O (Memory Mapping)” for details. Drivers set or clear this flag when the VIDIOC_QUERYBUF, VIDIOC_QBUF or VIDIOC_DQBUF ioctl is called. Set by the driver.

`V4L2_BUF_FLAG_QUEUED`

0x0002

Internally drivers maintain two buffer queues, an incoming and outgoing queue. When this flag is set, the buffer is currently on the incoming queue. It automatically moves to the outgoing queue after the buffer has been filled (capture devices) or displayed (output devices). Drivers set or clear this flag when the `VIDIOC_QUERYBUF` ioctl is called. After (successful) calling the `VIDIOC_QBUF` ioctl it is always set and after `VIDIOC_DQBUF` always cleared.

`V4L2_BUF_FLAG_DONE`

0x0004

When this flag is set, the buffer is currently on the outgoing queue, ready to be dequeued from the driver. Drivers set or clear this flag when the `VIDIOC_QUERYBUF` ioctl is called. After calling the `VIDIOC_QBUF` or `VIDIOC_DQBUF` it is always cleared. Of course a buffer cannot be on both queues at the same time, the `V4L2_BUF_FLAG_QUEUED` and `V4L2_BUF_FLAG_DONE` flag are mutually exclusive. They can be both cleared however, then the buffer is in "dequeued" state, in the application domain to say so.

`V4L2_BUF_FLAG_ERROR` | 0x0040 | When this flag is set, the buffer has been dequeued successfully, although the data might have been corrupted. This is recoverable, streaming may continue as normal and the buffer may be reused normally. Drivers set this flag when the `VIDIOC_DQBUF` ioctl is called.
`V4L2_BUF_FLAG_KEYFRAME` | 0x0008 | Drivers set or clear this flag when calling the `VIDIOC_DQBUF` ioctl. It may be set by video capture devices when the buffer contains a compressed image which is a key frame (or field), i. e. can be decompressed on its own.
`V4L2_BUF_FLAG_PFRAME` | 0x0010 | Similar to `V4L2_BUF_FLAG_KEYFRAME` this flags predicted frames or fields which contain only differences to a previous key frame.
`V4L2_BUF_FLAG_BFRAME` | 0x0020 | Similar to `V4L2_BUF_FLAG_PFRAME` this is a bidirectional predicted frame or field. [ooc tbd]
`V4L2_BUF_FLAG_TIMECODE` | 0x0100 | The `timecode` field is valid. Drivers set or clear this flag when the `VIDIOC_DQBUF` ioctl is called.
`V4L2_BUF_FLAG_PREPARED` | 0x0400 | The buffer has been prepared for I/O and can be queued by the application. Drivers set or clear this flag when the VIDIOC_QUERYBUF, VIDIOC_PREPARE_BUF, VIDIOC_QBUF or VIDIOC_DQBUF ioctl is called.
`V4L2_BUF_FLAG_NO_CACHE_INVALIDATE` | 0x0800 | Caches do not have to be invalidated for this buffer. Typically applications shall use this flag if the data captured in the buffer is not going to be touched by the CPU, instead the buffer will, probably, be passed on to a DMA-capable hardware unit for further processing or output.
`V4L2_BUF_FLAG_NO_CACHE_CLEAN` | 0x1000 | Caches do not have to be cleaned for this buffer. Typically applications shall use this flag for output buffers if the data in this buffer has not been created by the CPU but by some DMA-capable unit, in which case caches have not been used.
`V4L2_BUF_FLAG_TIMESTAMP_MASK` | 0xe000 | Mask for timestamp types below. To test the timestamp type, mask out bits not belonging to timestamp type by performing a logical and operation with buffer flags and timestamp mask.

`V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN`

0x0000

Unknown timestamp type. This type is used by drivers before Linux 3.9 and may be either monotonic (see below) or realtime (wall clock). Monotonic clock has been favoured in embedded systems whereas most of the drivers use the realtime clock. Either kinds of timestamps are available in user space via `clock_gettime(2)` using clock IDs `CLOCK_MONOTONIC` and `CLOCK_REALTIME`, respectively.

`V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC` | 0x2000 | The buffer timestamp has been taken from the `CLOCK_MONOTONIC` clock. To access the same clock outside V4L2, use `clock_gettime(2)` .
`V4L2_BUF_FLAG_TIMESTAMP_COPY` | 0x4000 | The CAPTURE buffer timestamp has been taken from the corresponding OUTPUT buffer. This flag applies only to mem2mem devices.

Table 3.5. enum v4l2_memory

`V4L2_MEMORY_MMAP` | 1 | The buffer is used for memory mapping I/O.
`V4L2_MEMORY_USERPTR` | 2 | The buffer is used for user pointer I/O.
`V4L2_MEMORY_OVERLAY` | 3 | [to do]
`V4L2_MEMORY_DMABUF` | 4 | The buffer is used for DMA shared buffer I/O.

### Timecodes

The v4l2_timecode structure is designed to hold a [SMPTE 12M] or similar timecode. (struct timeval timestamps are stored in struct v4l2_buffer field `timestamp`.)

Table 3.6. struct v4l2_timecode

__u32 | `type` | Frame rate the timecodes are based on, see Table 3.7, “Timecode Types”.
__u32 | `flags` | Timecode flags, see Table 3.8, “Timecode Flags”.
__u8 | `frames` | Frame count, 0 ... 23/24/29/49/59, depending on the type of timecode.
__u8 | `seconds` | Seconds count, 0 ... 59. This is a binary, not BCD number.
__u8 | `minutes` | Minutes count, 0 ... 59. This is a binary, not BCD number.
__u8 | `hours` | Hours count, 0 ... 29. This is a binary, not BCD number.
__u8 | `userbits`[4] | The "user group" bits from the timecode.

Table 3.7. Timecode Types

`V4L2_TC_TYPE_24FPS` | 1 | 24 frames per second, i. e. film.
`V4L2_TC_TYPE_25FPS` | 2 | 25 frames per second, i. e. PAL or SECAM video.
`V4L2_TC_TYPE_30FPS` | 3 | 30 frames per second, i. e. NTSC video.
`V4L2_TC_TYPE_50FPS` | 4 |
`V4L2_TC_TYPE_60FPS` | 5 |

Table 3.8. Timecode Flags

`V4L2_TC_FLAG_DROPFRAME` | 0x0001 | Indicates "drop frame" semantics for counting frames in 29.97 fps material. When set, frame numbers 0 and 1 at the start of each minute, except minutes 0, 10, 20, 30, 40, 50 are omitted from the count.
`V4L2_TC_FLAG_COLORFRAME` | 0x0002 | The "color frame" flag.
`V4L2_TC_USERBITS_field` | 0x000C | Field mask for the "binary group flags".
`V4L2_TC_USERBITS_USERDEFINED` | 0x0000 | Unspecified format.
`V4L2_TC_USERBITS_8BITCHARS` | 0x0008 | 8-bit ISO characters.

[19] Since no other Linux multimedia API supports unadjusted time it would be foolish to introduce here. We must use a universally supported clock to synchronize different media, hence time of day.

Prev | Up | Next
Asynchronous I/O | Home | Field Order