Quality metric performance on AR-DAVID dataset

The results are shown for six optical blending strategies, as explained in the paper. None of the metric was retrained on the AR-DAVID dataset. Only SROCC values are reported as, for most metrics, the correlations are too low to fit a logistic function and compute RMSE and PLCC. A liner function was fitted in the plots below for better visualization.



Metric/Variant SROCC
none average-lum pinhole pinhole-diplopic defocus defocus-diplopic
FovVideoVDP 1.2 → 0.3108 0.7573 0.7173 0.7173 0.709 0.709
ColorVideoVDP → 0.3507 0.7268 0.6318 0.6558 0.6329 0.6604
MDSI → 0.3648 0.6702 0.5838 0.5838 0.5879 0.5879
HaarPSI → 0.2975 0.5252 0.4699 0.4699 0.4751 0.4751
VMAF (v0.6.3) → 0.3143 0.4245 0.2866 0.3273 0.3794 0.3583
IW-SSIM → 0.03497 0.4657 0.2845 0.2845 0.2951 0.2951
GMSD → 0.03992 0.3527 0.3142 0.3142 0.3173 0.3173
MS-GMSD → 0.02833 0.3421 0.2951 0.2951 0.2965 0.2965
DSS → 0.04926 0.3094 0.2686 0.2686 0.2678 0.2678
MS-SSIM → 0.134 0.2459 0.1744 0.1744 0.1829 0.1829
DISTS → 0.03391 0.2695 0.226 0.226 0.1609 0.1609
PSNR (RGB) → 0.02232 0.2227 0.1761 0.1761 0.1805 0.1805
HDR-FLIP → 0.09468 0.1565 0.1539 0.1528 0.1508 0.1516
LPIPS (VGG) → 0.3581 0.02114 0.01184 0.01184 0.07819 0.07819

FovVideoVDP 1.2

Test: SROCC = 0.6719

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



ColorVideoVDP

Test: SROCC = 0.6208

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



MDSI

Test: SROCC = 0.5693

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



HaarPSI

Test: SROCC = 0.4547

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



VMAF (v0.6.3)

Test: SROCC = 0.3492

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



IW-SSIM

Test: SROCC = 0.281

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



GMSD

Test: SROCC = 0.2785

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



MS-GMSD

Test: SROCC = 0.2612

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



DSS

Test: SROCC = 0.24

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



MS-SSIM

Test: SROCC = 0.1826

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



DISTS

Test: SROCC = 0.1805

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



PSNR (RGB)

Test: SROCC = 0.1602

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



HDR-FLIP

Test: SROCC = 0.1434

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic



LPIPS (VGG)

Test: SROCC = 0.09574

Detailed report for:noneaverage-lumpinholepinhole-diplopicdefocusdefocus-diplopic