Steam gMAD Video Selection

Screening results from VQA2-dataset/steam-cvvdp-results, using 15 temporal pooling rules. Each original clip forms an independent content group; videos from different original clips are never paired.

Chinese report · Pilot pairs CSV · Pilot video assembly recipes

Inputs and selection method

The input contains 342 chunk-score files: 19 original clips × 3 chunks × 6 settings. Each setting combines one of two resolutions (720p or 1080p) with one of three frame rates (30, 60 or 120 fps). Each chunk lasts 1.5 seconds and has 180 proxy scores on a common 120 Hz timeline. Combining all three chunks gives 6³ = 216 candidate settings per original clip and 4,104 candidates in total.

gMAD alternates pooling rules as defender and attacker. Within each original clip, it searches for two candidates that the defender rates nearly equally but the attacker rates as differently as possible. The defender's predictions define 3 quantile bins. Both videos must fall in the same bin; equal predictions are kept together.

• Defender gap ≤ min(0.1, 5% × defender p95−p5 span).
• Attacker gap ≥ 10% × attacker p95−p5 span.
• Among feasible pairs in each bin, select the pair with the largest attacker gap.

There are 19 content groups × 15 defenders × 14 other attackers × 3 bins = 11,970 attack tasks. 11,299 tasks selected a valid pair, yielding 4,410 distinct gMAD pairs after deduplication. A task without a valid pair does not establish that the defender wins.

Pooling rules and score treatment

The 15 rules produce 12 distinct prediction groups on these candidates. The following rules have equivalent predictions for this run:

The 50th percentile is the median. The 5-second tail window covers each entire 4.5-second sequence; the duration-corrected cumulative-quality calculation also reduces to the overall mean at this duration.

Time policy: converted. The original 25 fps study durations are converted to 120 Hz while preserving their duration in seconds.

The four position-weighting rules use the three chunks: early dominance [1, 0, 0], late dominance [0, 0, 1], increasing weights [1/6, 2/6, 3/6], and decreasing weights [3/6, 2/6, 1/6].

Negative scores are excluded before pooling. This dataset contains 0 negative input scores, so 0 candidate sequences are affected. The input offset is 0; retained scores range from 3.204 to 10.000. Every candidate has 540 scores, corresponding to 4.5 seconds.

Pilot selection

The pilot contains 266 pairs involving 440 distinct videos: 190 gMAD pairs, 38 random pairs and 38 diagnostic pairs. Each original clip contributes 10 gMAD pairs, 2 random pairs and 2 diagnostic pairs.

• gMAD pairs are chosen greedily to cover additional effective defender–attacker–bin combinations, with normalized attacker gap used to break coverage ties.
• Random pairs include one pair of fixed-setting videos and one pair where at least one video changes settings. They need not meet the gMAD gap criteria.
• Diagnostic pairs compare reversed setting orders, or fixed against varying settings with similar mean scores.

The pilot is a subset for an initial study. The full gMAD list contains all distinct pairs selected by successful attack tasks. Video A and Video B are list labels; participant presentation order should be randomized.

Pilot pair list

266 of 266 pairs shown. The settings in each video column follow chunk order 1 → 2 → 3.

Verification and sensitivity

Saved validation status: passed. Input CSV and metadata hashes were verified. All 11,299 selected tasks were independently checked for the same content group, the same defender quality bin, the required score gaps and the maximum feasible attacker gap. Pilot recipes cover all 440 pilot video IDs.

These results are a first screening from concatenated chunk-score sequences. Assemble the selected chunks into actual 4.5-second videos and recompute full-video CVVDP before finalizing experimental stimuli, particularly around chunk transitions. Human evaluation has not yet been performed; this report does not establish a perceptual ranking of pooling rules.

Result files

• Pilot pairs · All distinct gMAD pairs
• Pilot assembly recipes · All selected assembly recipes
• Candidate predictions · Attack tasks · Compact task design
• Parameters and input hashes · Summary · Validation · Sensitivity results
• Selection command · Selection log