GAIM240 Temporal Pooling

SROCC by rule (216 videos)

Both charts share one axis. VMAF's own pooling is the arithmetic mean of the per-frame scores (the pooled_metrics.mean that libvmaf reports), so in the VMAF chart the reference line equals the mean. last_frames_mean and cumulative_quality equal mean on 5 s clips and are not drawn; their values are in the table view. Hover over a row, or Tab to it, to see its parameters and PLCC.

Reading the results

• cvvdp: the plain per-frame mean (0.566) is already higher than cvvdp's own pooling (0.541). Rules that weight the later part of the video more do better: late_dominance 0.625, increasing_weights 0.585, exp_weighted_minkowski 0.583.
• Rules that emphasise the worst frames fall below the mean on cvvdp: percentile 0.487, kth_percentile 0.509, worst_window 0.552, hysteresis 0.554.
• VMAF: no rule clearly beats the mean (0.437). Rules that emphasise the worst frames lose the most: percentile 0.321, hysteresis 0.330, kth_percentile 0.346.
• bistro_exterior alone (24 videos): on cvvdp the mean scores 0.770 and late_dominance 0.746, so the ranking differs from the 216-video result. With few videos, the order of the rules changes.
• last_frames_mean and cumulative_quality use windows (5 s and 50 to 60 s in the source papers) that are not shorter than a 5 s clip, so they equal the mean.

Caveats

• The 216 videos come from 9 scenes. No significance test or confidence interval was computed, so rules that differ by 0.01 to 0.02 cannot be ranked against each other.
• Rule parameters are the paper defaults. Parameters given in frames at 25 fps were converted to the same duration in seconds (multiplied by 9.6 at 240 fps). Nothing was tuned on GAIM240. For late_dominance, the video is split in half and the split point was not optimised.
• SROCC is computed over all scenes together. PLCC is computed without a non-linear fit.
• Minkowski-type rules use p = −2 on quality scores, so that the worst frames get the largest weight.
• The per-frame cvvdp JOD is obtained by pooling cvvdp's per-frame features over bands and channels. Pooling the per-frame values over frames the way cvvdp does reproduces the video JOD it reports (checked by an assertion in the script). VMAF keeps the original metric_calibration settings (vmaf_b_v0.6.3, 10-bit 4:4:4).
• Rules not run: hammerstein_wiener (the papers give no fitted coefficients) and the two frame-time rules (their input is not a quality score).

Where the results are

All paths are on iridium. The results exist in two places: the working directory of metric_calibration (files there belong to the local branch temporal-pooling) and a copy in VQA2.