Each broadcaster - traditional or web caster - must deal with rapidly changing varieties of programming, new video processing algorithms, and new compression algorithms. Video processing and compression companies continuously invent sophisticated ways to reduce the huge bandwidth requirements to manageable levels. How can broadcasters know if a new algorithm is better than their current choice?
Broadcasters invite the various video processing and compression companies into their R&D facilities, and perform side-by-side tests also known as a "bake-off". Each vendor starts with the same source material, and does their best to reduce the bandwidth while keeping the video quality high.
The broadcaster shows the results to a group of experts and asks them, which one is the best. This is termed subjective video analysis, and it measures the overall perceived video quality. The most commonly used video quality evaluation method is the Mean Opinion Score (MOS), recommended by the ITU. It consists in having several experts viewing a known distorted video sequences in order to rate its quality, according to a predefined quality scale. By doing this the expert viewers are trained to build a mapping between the quality scale and a set of processed video sequences. After the "training" is complete, the subjects are then asked to rate the new video processing algorithms.
Simply stated, the test setup is
- Start with a known video sequence.
- New Video Processing system alters the video sequence.
- Display the original and processed video sequences.
- Bring in experts to subjectively vote.
Complexity arises as
- New Video Processing systems may need new equipment to playback the video sequences.
- The original and processed video sequences should be displayed in random orders.
- Expert viewers are expensive and do not produce repeatable results.
Easier Solution To streamline the process, equipment for video quality testing needs to be defined, which can capture, play, and analyze any two video sequences. Further, as new input/output modules are continuously under development, the test equipment should use an open-architecture approach to ease upgradeability.
The following are the key attributes of a robust video quality testing tools.
- Allow a way to import video sequences regardless of their file type - i.e. AVI, QuickTime, Raw, Video Editor, MPEG, etc.
- Serve video sequences to the encoder and/or video processing unit using SDI, Component, DVB-ASI or DVI.
- Capture the output of the encoder or the encoder/decoder pair.
- Convert all video sequences to user-selectable resolution, bit depth, and color format so that they can be displayed multiple viewing modes on the same display.
- Support 8 and 10-bit data paths with upgradeability to future 16-bit modes.
- Store the video sequences as frames (fields) so that they can be played at any rate.
- Allow multiple playing modes such as play, shuttle, jog, pause, zoom and pan.
- Apply objective metrics to the video sequences to score the video frame-by-frame, and log/graph the results for easy analysis.
- Export pieces of video sequences to further analyze off-line.
Case Examples Using this setup, broadcasters can compare competitor's image processing algorithms, their customer's new algorithms with respect to their old, and/or the compressed video with respect to the original. The video sequences can be programmatically displayed in multiple viewing modes or sent to 2 displays for subjective assessment. At the same time, multiple objective metrics are calculated, graphed, and logged for repeatable, quantitative objective scoring. Furthermore, the entire process can be fully automated.
A Solution/System Integrator creates a broadcast solution using encoder, network, video processing, VOD, conditional access, and set-top box providers. They market and sell their solution to Broadcasters or Service Providers. So they must provide the test results based on the above setup in a written report, and they would like to show their Broadcast customer the video results. At times they can get the broadcast to come to their facility to view the solution, while at other times they must show the solution results within the broadcast facility. Thus, they have the additional requirement for a
- Portable solution to show side-by-side comparisons to the Broadcaster or Service Provider.