Computer Laboratory


ColorVideoVDP: A visual difference predictor for image, video and display distortions

Rafał K. Mantiuk(1), Param Hanji(1), Maliha Ashraf(1), Yuta Asano(2), and Alexandre Chapiro(2).

(1)University of Cambridge, (2)Meta Reality Labs

Presented at SIGGRAPH 2024, Technical Papers

ColorVideoVDP predicts the visibility of distortions for a pair of test and reference videos (or images) as seen on a display with a provided specification. The predictions are represented as a single quality value in Just-Objectionable-Difference (JOD) units, a distortion map video, and a distogram, which visualizes the distortions over time, separately for each channel and spatial frequency band.


ColorVideoVDP is a video and image quality metric that models spatial and temporal aspects of vision for both luminance and color. The metric is built on novel psychophysical models of chromatic spatiotemporal contrast sensitivity and cross-channel contrast masking. It accounts for the viewing conditions, geometric, and photometric characteristics of the display. It was trained to predict common video-streaming distortions (e.g., video compression, rescaling, and transmission errors) and also 8 new distortion types related to AR/VR displays (e.g., light source and waveguide non-uniformities). To address the latter application, we collected our novel XR-Display-Artifact-Video quality dataset (XR-DAVID), comprised of 336 distorted videos. Extensive testing on XR-DAVID, as well as several datasets from the literature, indicate a significant gain in prediction performance compared to existing metrics. ColorVideoVDP opens the doors to many novel applications that require the joint automated spatiotemporal assessment of luminance and color distortions, including video streaming, display specification, and design, visual comparison of results, and perceptually-guided quality optimization.


  • Paper:
    ColorVideoVDP: A visual difference predictor for image, video and display distortions.
    Rafal K. Mantiuk, Param Hanji, Maliha Ashraf, Yuta Asano, Alexandre Chapiro.
    In SIGGRAPH 2024 Technical Papers, Article 129
    [DOI] [paper PDF]
  • Supplementary document [PDF]
  • Code [Github]
  • XR-DAVID dataset


Related projects

  • FovVideoVDP - Foveated Video Visual Difference Predictor
  • DPVM - Deep Photometric Visual Metric
  • HDR-VDP - A Visual Difference Predictor for High Dynamic Range Images
  • castleCSF - A Contrast Sensitivity Function of Color, Area, Spatio-Temporal frequency, Luminance and Eccentricity - models contrast sensitivity in ColorVideoVDP
  • ASAP - Active Sampling for Pairwise Comparisons - used to efficiently collect XR-DAVID dataset
  • pwcmp - Bayesian pairwise comparison scaling - used to scale XR-DAVID subjective responses