Temporal pooling rules vs MOS (DSIS pilot)
7 participants, 84 videos, 8 scenes; no participant was rejected. Within each scene, all of its videos (every shot together) are ranked by MOS and by the rule's score, and the Spearman correlation (SRCC) between the two rankings is computed. The 8 scene SRCCs are averaged in Fisher z space: equal-weighted gives every scene the same weight; weighted uses n − 3 (number of videos minus 3), so scenes with more videos count more.
1. Parameters tuned on makeway, tested on the other scenes
Each rule's parameters were chosen by grid search to maximise SRCC against MOS on makeway's 19 videos, then frozen and evaluated on the 65 videos of the other 7 scenes. The dashed line is the plain mean under the same measure. PSNR-Y is pooled in the MSE domain, as in section 2.
CVVDP per-frame JOD
VMAF per frame
PSNR-Y per frame
Parameter values: default vs tuned on makeway
Time parameters in milliseconds, with 120 Hz frames in brackets. Tuning on a millisecond grid (tune_pooling_makeway.py --unit ms) selects the same values.
| Rule | Parameter | Default | Tuned, CVVDP | Tuned, VMAF | Tuned, PSNR-Y |
|---|---|---|---|---|---|
| minkowski | p | 2 | 2 | 8 | 1 |
| exp_weighted_minkowski | p | 2 | 4 | 4 | 4 |
| τ | 4000 ms (480 frames) | 16000 ms (1920 frames) | 16000 ms (1920 frames) | 16000 ms (1920 frames) | |
| percentile | p (%) | 5 | 100 | 100 | 100 |
| kth_percentile | k | 50 | 90 | 60 | 60 |
| last_frames_mean | last | 5000 ms (600 frames) | 5000 ms (600 frames) | 5000 ms (600 frames) | 5000 ms (600 frames) |
| worst_window | window | 4000 ms (480 frames) | 1500 ms (180 frames) | 750 ms (90 frames) | 4500 ms (540 frames) |
| hysteresis | τ | 2000 ms (240 frames) | 100 ms (12 frames) | 2000 ms (240 frames) | 2000 ms (240 frames) |
| α | 0.8 | 0 | 0 | 0 | |
| early_dominance | segments | 3 | 2 | 2 | 2 |
| late_dominance | segments | 3 | 2 | 3 | 3 |
| increasing_weights | segments | 3 | 2 | 4 | 2 |
| decreasing_weights | segments | 3 | 2 | 2 | 4 |
Show table (makeway SRCC, held-out SRCC, bootstrap CI)
Gain in pooled held-out SRCC, 95% CI from 2000 paired bootstrap resamples of the 65 videos.
2. Scene-averaged SRCC per rule
The same rules, with the same parameters, applied to three per-frame scores: CVVDP JOD, VMAF and PSNR-Y. Each chart is sorted by the weighted average and all share one x-axis. Small scenes (heros3 and ratchet, 5 videos each) inflate the equal-weighted average, so the weighted average is the one to read.
CVVDP per-frame JOD
Click a row to show that rule in the scatter plots of section 4.
VMAF per frame
PSNR-Y per frame
The rules pool per-frame MSE and the result is converted to dB (mean = PSNR of the mean MSE), so frames identical to the reference (60 dB in libvmaf) do not inflate the score.
3. SRCC per scene and rule
Top: rules on the x-axis (sorted by the weighted average); each dot is one scene's SRCC for that rule. Each scene keeps the same horizontal offset within every column. Click a scene in the legend to isolate it.
* mean is identical to last_frames_mean and cumulative_quality, and median to kth_percentile, on these videos.
Bottom: the same values as a heatmap; darker means higher correlation. Under each scene is its number of videos; the fewer the videos, the less reliable the cell.
4. Per scene: rule score vs MOS
Each dot is one video; the vertical line is the 95% confidence interval of its MOS, and colour marks the shot within the scene. The x-axis is the JOD the rule gives; its range differs between scenes.