What is the color space (format) used by Media Recorder?

In the Media Recorder Settings Window, the Format column represents the color format of the signal that is used as input for the Media Recorder. In Media Recorder 2 this column is hidden by default (right-click the header to show). Each color format has its own method of reducing the color information of the video file.


The RGB color model
The RGB color model describes what kind of light needs to be emitted to produce a given color. RGB stores individual values for red, green and blue. When the RGB color model is used, each pixel contains color information on the components Red, Green and Blue.

Chroma subsampling
Chroma subsampling is a way to divide the color information from the RGB model into black and white (luma) and color (chroma). The human eye is more sensitive to differences in brightness (luminance) than for color (chrominance). Therefore, part of the information on chrominance can be left out without our eyes being able to detect that. The figure below shows different methods for leaving out chrominance information, while all the luminance information is stored. In the picture on the right, every pixel contains all information on both luminance and chrominance. 
Chroma subsampling is often referred to as Y’CbCr subsampling. Y contains the information for luminance, Cr contains the information for the color red and Cb contains the information blue.

The figure above shows different schemes for Chroma subsampling. The upper row shows the luminance information for every pixel. Hence for every pixel a value for black or white is stored. The middle row shows the chroma subsampling for the colors red and blue. In the picture on the left, for every four pixels only one pixel is stored. The bottom row shows the result of Chroma subsampling. The numbers (J: a: b) indicate:
J: the number of pixels in every sample
a: number of chrominance pixels that are stored for every J pixels in the first row.
b: the number of additional pixels that are stored for every J pixels in the second row.
In the picture on the right, every pixel contains all information on both luminance and chrominance.

There is a number of color format models that are varieties of the Chroma subsampling method. One is YUV, in which Y stands for luminance and U and V stand for the chrominance components. Another one is Y’CrCb in which Y stands for luminance and Cr and Cb for chrominance. UYVY is a color format with a chroma subsampling method similar to Y’CbCr. 


Bayer filter and RAW format
Many cameras use a Bayer filter to transform the color information of the video images. A Bayer filter, is a filter that is placed over the photo sensors of the camera. With a Bayer filter, every pixel gets information for only one color, Red, Blue or Green. The other color information is discarded.  The figure below shows how the colors are arranged in a Bayer filter.
The color format of a video file in which a Bayer filter is applied, is called the RAW format. To obtain a full-color image, red, green, and blue values for each pixel are calculated. For this purpose, various demosaicing algorithms are available.

Color interpolation
In addition to converting the color information to RAW format, some cameras have the ability to use color interpolation. In this method, every pixel has only one color, like in the RAW format. In addition to this, the color of every pixel is interpolated using the color from all the neighboring pixels.
Monochrome formats
The formats Y800 and Y8 are monochrome variants of the YUV format. These formats only sample the monochrome Y component of the frames. The Y800 codec supplied by some camera manufacturers can decode the color information. However, the codecs that are supplied with The Media Recorder cannot decode the color of the Y800 format.


Preferred formats of Media Recorder
The preferred formats of the Media Recorder are RGB24 and UYVY, in which RGB24 is preferred over UYVY. If a device does not support one of these formats, the Media Recorder uses the default format from the device. If the Media Recorder is used in combination with the Picolo U4 H.264 board, the format is H.264, which is the format of the video file created by the Media Recorder. No color format is used.

JPEG compression
In video files with MJPEG format, each frame is compressed using a similar algorithm as for still images (JPEG compression). JPEG compression uses mathematical tricks such as Huffman compression so that the information about the matrix of pixels is stored optimally. JPEG compression works optimally with smooth gradients and is less good with sharply contrasting lines.
 
The figure above shows JPEG-compression. The original bar on the left has a gradient of many colors and this is highly compressed to a few colors on the right.