ColorVideoVDP - ablation studies on contrast and masking models

The results in the plot below are reported for the testing data only. Only UPIQ and XR-DAVID were used for training (unless stated otherwise). The training set was also used to fit the JOD regression. The detailed reports (if present) show the entire dataset (both training and testing parts).



Table shows the results for test data

Metric/Variant RMSE PLCC SROCC
UPIQ XR-DAVID KADID-10k LIVEVQA LIVEHDR Average(1) UPIQ XR-DAVID KADID-10k LIVEVQA LIVEHDR Average(2) UPIQ XR-DAVID KADID-10k LIVEVQA LIVEHDR Average(3)
Multiplicative contrast - mutual masking → 0.6073 0.5944 0.9262 0.8585 0.3169 0.6607 0.9 0.8782 0.8595 0.756 0.9357 0.8773 0.8759 0.8913 0.8605 0.7412 0.9244 0.8694
Additive contrast - mutual masking → 0.7707 0.8996 1.091 1.118 0.6643 0.9086 0.8294 0.7836 0.798 0.7157 0.8339 0.7957 0.8083 0.7966 0.7982 0.7189 0.8565 0.7998
Multiplicative contrast - contrast transducer → 0.8853 0.6103 1.091 0.9454 1.033 0.9131 0.7716 0.8787 0.7984 0.6931 0.814 0.7991 0.7739 0.8663 0.8091 0.6782 0.8533 0.8052
Additive contrast - contrast transducer → 1.029 0.9087 1.326 0.9839 0.7161 0.9927 0.666 0.7582 0.6813 0.6839 0.5776 0.6775 0.6806 0.72 0.6848 0.6804 0.6437 0.6827
Additive contrast - similarity → 1.344 1.278 1.716 1.313 0.8173 1.294 0.2176 -0.003649 0.3182 -0.2236 0.1312 0.09008 0.04474 -0.169 0.2481 -0.2154 0.1703 0.01614
Multiplicative contrast - similarity → 1.386 1.268 1.81 1.311 0.9342 1.342 -0.3098 0.002515 -0.001624 0.04326 -0.004255 -0.05601 -0.3733 0.02586 -0.1083 0.1294 -0.03046 -0.07493

Table shows the results for all data

Metric/Variant RMSE PLCC SROCC
UPIQ XR-DAVID KADID-10k LIVEVQA LIVEHDR Average(1) UPIQ XR-DAVID KADID-10k LIVEVQA LIVEHDR Average(2) UPIQ XR-DAVID KADID-10k LIVEVQA LIVEHDR Average(3)
Multiplicative contrast - mutual masking → 0.67 0.7768 0.9262 0.8585 0.6022 0.7667 0.8956 0.7843 0.8595 0.756 0.8639 0.8392 0.8876 0.8328 0.8605 0.7412 0.9071 0.8548
Additive contrast - mutual masking → 0.9076 1.042 1.091 1.118 0.801 0.9917 0.7986 0.6331 0.798 0.7157 0.6618 0.7286 0.8029 0.697 0.7982 0.7189 0.7979 0.7666
Multiplicative contrast - contrast transducer → 0.9482 0.711 1.091 0.9454 1.345 1.008 0.7767 0.8187 0.7984 0.6931 0.7383 0.7686 0.7994 0.8268 0.8091 0.6782 0.8048 0.7885
Additive contrast - contrast transducer → 1.18 0.9973 1.326 0.9839 0.9375 1.085 0.6213 0.6255 0.6813 0.6839 0.5354 0.6324 0.663 0.6319 0.6848 0.6804 0.5945 0.6522
Additive contrast - similarity → 1.473 1.262 1.716 1.313 0.8622 1.325 0.2082 0.09843 0.3182 -0.2236 0.3402 0.1521 0.006221 0.03118 0.2481 -0.2154 0.3197 0.08049
Multiplicative contrast - similarity → 1.515 1.239 1.81 1.311 0.9699 1.369 -0.3064 -0.04445 -0.001624 0.04326 0.0003522 -0.06371 -0.2817 -0.03349 -0.1083 0.1294 0.01765 -0.05671

(1)Average RMSE is computed as an average of RMSEs of individual datasets so that each dataset has the same influence on the average RMSE regardless of the number of conditions it contains.

(2-3)Average correlation coeffcients are computed by averaging individual correlation coefficients (in the Fisher's transorm space).


Multiplicative contrast - mutual masking

Test: RMSE = 0.6607 PLCC = 0.8773 SROCC = 0.8694

Detailed report for:UPIQXR-DAVIDKADID-10kLIVEVQALIVEHDR



Additive contrast - mutual masking

Test: RMSE = 0.9086 PLCC = 0.7957 SROCC = 0.7998

Detailed report for:UPIQXR-DAVIDKADID-10kLIVEVQALIVEHDR



Multiplicative contrast - contrast transducer

Test: RMSE = 0.9131 PLCC = 0.7991 SROCC = 0.8052

Detailed report for:UPIQXR-DAVIDKADID-10kLIVEVQALIVEHDR



Additive contrast - contrast transducer

Test: RMSE = 0.9927 PLCC = 0.6775 SROCC = 0.6827

Detailed report for:UPIQXR-DAVIDKADID-10kLIVEVQALIVEHDR



Additive contrast - similarity

Test: RMSE = 1.294 PLCC = 0.09008 SROCC = 0.01614

Detailed report for:UPIQXR-DAVIDKADID-10kLIVEVQALIVEHDR



Multiplicative contrast - similarity

Test: RMSE = 1.342 PLCC = -0.05601 SROCC = -0.07493

Detailed report for:UPIQXR-DAVIDKADID-10kLIVEVQALIVEHDR