Ablation: pooling



Metric/Variant RMSE PLCC SROCC
FovDots UPIQ DeepFovea LIVE-FBT-FCVR Average(1) FovDots UPIQ DeepFovea LIVE-FBT-FCVR Average(2) FovDots UPIQ DeepFovea LIVE-FBT-FCVR Average(3)
Pixels + temporal-channels → 0.8938 0.7696 0.9376 0.6481 0.8123 0.7873 0.8599 0.8322 0.728 0.8554 0.8148 0.83 0.8413 0.7254 0.8287
Spatial-channels → 0.9213 0.7484 0.9469 0.6453 0.8155 0.7787 0.8682 0.8231 0.74 0.8633 0.8226 0.8485 0.8471 0.7347 0.8468
Pixels + frames → 0.8651 0.7949 0.9394 0.6878 0.8218 0.805 0.8493 0.8224 0.7125 0.8451 0.8328 0.8182 0.8402 0.7118 0.8174
Temporal-channels → 0.9567 0.7093 0.9752 0.6518 0.8233 0.7591 0.882 0.8097 0.7231 0.8758 0.7979 0.8489 0.8291 0.7206 0.8461
Frames → 0.9026 0.7708 0.9685 0.6793 0.8303 0.7931 0.859 0.8057 0.7047 0.854 0.8313 0.8269 0.8292 0.7027 0.8255
Pixels → 0.8837 0.8147 0.9449 0.6841 0.8319 0.7938 0.8409 0.8262 0.7113 0.837 0.8284 0.8112 0.8401 0.711 0.8104
Pixels + spatial-channels → 0.8977 0.838 0.9438 0.651 0.8326 0.7877 0.8307 0.8571 0.7451 0.8277 0.8114 0.8204 0.8493 0.7482 0.8195
Pixels + spat.-chan. + temp.-chan. → 0.9033 0.8438 0.9259 0.6773 0.8376 0.7944 0.8286 0.8467 0.739 0.8255 0.8293 0.8238 0.8457 0.7424 0.8228
Pixels + spatial-channels → 0.9392 0.8841 0.9581 0.6776 0.8647 0.7656 0.8093 0.8752 0.7392 0.8065 0.7828 0.8058 0.8486 0.7409 0.8044

(1)Average RMSE is computed as an average of RMSEs of individual datasets so that each dataset has the same influence on the average RMSE regardless of the number of conditions it contains.

(2-3)Average correlation coeffcients are computed for the consolidated dataset consisting of all four individual datasets. It means that the average correlation coefficients are dominated by the performance of UPIQ, which contains over 4000 conditions. For that reason, RMSE is more indicative of metric performance.


Pixels + temporal-channels

RMSE = 0.8123 PLCC = 0.8554 SROCC = 0.8287

JOD regression: Q = 10 -0.0129523 * M^0.596953

Spatial-channels

RMSE = 0.8155 PLCC = 0.8633 SROCC = 0.8468

JOD regression: Q = 10 -0.0671421 * M^0.371226

Pixels + frames

RMSE = 0.8218 PLCC = 0.8451 SROCC = 0.8174

JOD regression: Q = 10 -0.059422 * M^0.458154

Temporal-channels

RMSE = 0.8233 PLCC = 0.8758 SROCC = 0.8461

JOD regression: Q = 10 -0.103737 * M^0.418913

Frames

RMSE = 0.8303 PLCC = 0.854 SROCC = 0.8255

JOD regression: Q = 10 -0.143542 * M^0.385292

Pixels

RMSE = 0.8319 PLCC = 0.837 SROCC = 0.8104

JOD regression: Q = 10 -0.11692 * M^0.402183

Pixels + spatial-channels

RMSE = 0.8326 PLCC = 0.8277 SROCC = 0.8195

JOD regression: Q = 10 -0.00826105 * M^0.484441

Pixels + spat.-chan. + temp.-chan.

RMSE = 0.8376 PLCC = 0.8255 SROCC = 0.8228

JOD regression: Q = 10 -0.0193776 * M^0.358818

Pixels + spatial-channels

RMSE = 0.8647 PLCC = 0.8065 SROCC = 0.8044

JOD regression: Q = 10 -0.00818974 * M^0.425018