Image-space metric performance on the subjective BRDF quality dataset
The report shows the results for the following performance indices:
- SROCC — Spearman's Rank-Order Correlation Coefficient
- PLCC — Pearsons's Linear Correlation Coefficient
- KROCC — Kendall's Rank-Order Correlation Coefficient
- RMSE — Root Mean Squared Error (in JODs)
You can click on the BRDF-Quality links below to see the detailed scatter plots showing the relation between metric predictions and the BRDF subjective quality.
| Metric/Variant |
SROCC |
RMSE |
KROCC |
PLCC |
|---|
| BRDF-Quality |
BRDF-Quality |
BRDF-Quality |
BRDF-Quality |
|---|
| DeltaE ITP →
|
0.8463 |
1.038 |
0.653 |
0.841 |
| MDSI →
|
0.8069 |
1.051 |
0.6172 |
0.8369 |
| TOPIQ →
|
0.8166 |
1.075 |
0.6183 |
0.8286 |
| LPIPS (AlexNet) →
|
0.8188 |
1.079 |
0.6255 |
0.8267 |
| HaarPSI →
|
0.809 |
1.157 |
0.6187 |
0.8133 |
| LPIPS (VGG) →
|
0.7865 |
1.135 |
0.591 |
0.8064 |
| ColorVideoVDP (v0.4.2) →
|
0.8075 |
1.153 |
0.609 |
0.7995 |
| PSNR-RGB →
|
0.7996 |
1.178 |
0.6027 |
0.7991 |
| VSI →
|
0.789 |
1.173 |
0.6074 |
0.7935 |
| PieAPP →
|
0.7736 |
1.177 |
0.5933 |
0.7902 |
| HDR-FLIP →
|
0.7983 |
1.193 |
0.6034 |
0.7831 |
| DISTS →
|
0.7718 |
1.207 |
0.5714 |
0.7771 |
| FSIMc →
|
0.7793 |
1.233 |
0.5887 |
0.776 |
| MS-SSIM →
|
0.7436 |
1.338 |
0.5523 |
0.7263 |
| NLPD →
|
0.7173 |
1.33 |
0.5287 |
0.7207 |
| SSIM (luma) →
|
0.7013 |
1.357 |
0.5108 |
0.7108 |
| IW-SSIM →
|
0.7109 |
1.367 |
0.5239 |
0.7084 |
| PSNR-Y →
|
0.7165 |
1.359 |
0.5188 |
0.7058 |
| MS-GMSD →
|
0.7049 |
1.367 |
0.5199 |
0.7017 |
| HDR-VDP-3 (v3.0.7) →
|
0.7176 |
1.372 |
0.5302 |
0.699 |
| GMSD →
|
0.6959 |
1.38 |
0.5099 |
0.6944 |
| Hybrid Delta E L*a*b* →
|
0.7125 |
1.399 |
0.5195 |
0.6841 |
| CIEDE2000 →
|
0.7142 |
1.402 |
0.5198 |
0.6827 |
| CIEDE2000 spatial →
|
0.7117 |
1.406 |
0.5139 |
0.6803 |
| DSS →
|
0.7036 |
1.416 |
0.5173 |
0.6748 |
| STRRED →
|
0.665 |
1.441 |
0.4737 |
0.66 |
| AHIQ →
|
0.6669 |
1.449 |
0.4644 |
0.6556 |
| DeltaE ITP spatial →
|
0.6822 |
1.451 |
0.4852 |
0.6539 |
| VMAF (v0.6.3) →
|
0.5987 |
1.469 |
0.4225 |
0.6428 |
| VIF →
|
0.6309 |
1.513 |
0.4479 |
0.6148 |
| CKDN →
|
0.3496 |
1.725 |
0.2469 |
0.4375 |
| BRISQUE →
|
0.1818 |
1.839 |
0.1205 |
0.2857 |
| NIQE →
|
0.04976 |
1.91 |
0.03681 |
0.09065 |
DeltaE ITP
Test:
SROCC = 0.8463 RMSE = 1.038 KROCC = 0.653 PLCC = 0.841 Detailed report for:BRDF-Quality
MDSI
Test:
SROCC = 0.8069 RMSE = 1.051 KROCC = 0.6172 PLCC = 0.8369 Detailed report for:BRDF-Quality
TOPIQ
Test:
SROCC = 0.8166 RMSE = 1.075 KROCC = 0.6183 PLCC = 0.8286 Detailed report for:BRDF-Quality
LPIPS (AlexNet)
Test:
SROCC = 0.8188 RMSE = 1.079 KROCC = 0.6255 PLCC = 0.8267 Detailed report for:BRDF-Quality
HaarPSI
Test:
SROCC = 0.809 RMSE = 1.157 KROCC = 0.6187 PLCC = 0.8133 Detailed report for:BRDF-Quality
LPIPS (VGG)
Test:
SROCC = 0.7865 RMSE = 1.135 KROCC = 0.591 PLCC = 0.8064 Detailed report for:BRDF-Quality
ColorVideoVDP (v0.4.2)
Test:
SROCC = 0.8075 RMSE = 1.153 KROCC = 0.609 PLCC = 0.7995 Detailed report for:BRDF-Quality
PSNR-RGB
Test:
SROCC = 0.7996 RMSE = 1.178 KROCC = 0.6027 PLCC = 0.7991 Detailed report for:BRDF-Quality
VSI
Test:
SROCC = 0.789 RMSE = 1.173 KROCC = 0.6074 PLCC = 0.7935 Detailed report for:BRDF-Quality
PieAPP
Test:
SROCC = 0.7736 RMSE = 1.177 KROCC = 0.5933 PLCC = 0.7902 Detailed report for:BRDF-Quality
HDR-FLIP
Test:
SROCC = 0.7983 RMSE = 1.193 KROCC = 0.6034 PLCC = 0.7831 Detailed report for:BRDF-Quality
DISTS
Test:
SROCC = 0.7718 RMSE = 1.207 KROCC = 0.5714 PLCC = 0.7771 Detailed report for:BRDF-Quality
FSIMc
Test:
SROCC = 0.7793 RMSE = 1.233 KROCC = 0.5887 PLCC = 0.776 Detailed report for:BRDF-Quality
MS-SSIM
Test:
SROCC = 0.7436 RMSE = 1.338 KROCC = 0.5523 PLCC = 0.7263 Detailed report for:BRDF-Quality
NLPD
Test:
SROCC = 0.7173 RMSE = 1.33 KROCC = 0.5287 PLCC = 0.7207 Detailed report for:BRDF-Quality
SSIM (luma)
Test:
SROCC = 0.7013 RMSE = 1.357 KROCC = 0.5108 PLCC = 0.7108 Detailed report for:BRDF-Quality
IW-SSIM
Test:
SROCC = 0.7109 RMSE = 1.367 KROCC = 0.5239 PLCC = 0.7084 Detailed report for:BRDF-Quality
PSNR-Y
Test:
SROCC = 0.7165 RMSE = 1.359 KROCC = 0.5188 PLCC = 0.7058 Detailed report for:BRDF-Quality
MS-GMSD
Test:
SROCC = 0.7049 RMSE = 1.367 KROCC = 0.5199 PLCC = 0.7017 Detailed report for:BRDF-Quality
HDR-VDP-3 (v3.0.7)
Test:
SROCC = 0.7176 RMSE = 1.372 KROCC = 0.5302 PLCC = 0.699 Detailed report for:BRDF-Quality
GMSD
Test:
SROCC = 0.6959 RMSE = 1.38 KROCC = 0.5099 PLCC = 0.6944 Detailed report for:BRDF-Quality
Hybrid Delta E L*a*b*
Test:
SROCC = 0.7125 RMSE = 1.399 KROCC = 0.5195 PLCC = 0.6841 Detailed report for:BRDF-Quality
CIEDE2000
Test:
SROCC = 0.7142 RMSE = 1.402 KROCC = 0.5198 PLCC = 0.6827 Detailed report for:BRDF-Quality
CIEDE2000 spatial
Test:
SROCC = 0.7117 RMSE = 1.406 KROCC = 0.5139 PLCC = 0.6803 Detailed report for:BRDF-Quality
DSS
Test:
SROCC = 0.7036 RMSE = 1.416 KROCC = 0.5173 PLCC = 0.6748 Detailed report for:BRDF-Quality
STRRED
Test:
SROCC = 0.665 RMSE = 1.441 KROCC = 0.4737 PLCC = 0.66 Detailed report for:BRDF-Quality
AHIQ
Test:
SROCC = 0.6669 RMSE = 1.449 KROCC = 0.4644 PLCC = 0.6556 Detailed report for:BRDF-Quality
DeltaE ITP spatial
Test:
SROCC = 0.6822 RMSE = 1.451 KROCC = 0.4852 PLCC = 0.6539 Detailed report for:BRDF-Quality
VMAF (v0.6.3)
Test:
SROCC = 0.5987 RMSE = 1.469 KROCC = 0.4225 PLCC = 0.6428 Detailed report for:BRDF-Quality
VIF
Test:
SROCC = 0.6309 RMSE = 1.513 KROCC = 0.4479 PLCC = 0.6148 Detailed report for:BRDF-Quality
CKDN
Test:
SROCC = 0.3496 RMSE = 1.725 KROCC = 0.2469 PLCC = 0.4375 Detailed report for:BRDF-Quality
BRISQUE
Test:
SROCC = 0.1818 RMSE = 1.839 KROCC = 0.1205 PLCC = 0.2857 Detailed report for:BRDF-Quality
NIQE
Test:
SROCC = 0.04976 RMSE = 1.91 KROCC = 0.03681 PLCC = 0.09065 Detailed report for:BRDF-Quality
