Projects

Streaming of rendered content with adaptive frame rate and resolution

Yaru Liu, Joseph March and Rafał K. Mantiuk.

University of Cambridge

Presented at SIGGRAPH 2026, Conference Proceedings

Streaming of rendered content with adaptive frame rate and resolution
Motivated by the goal of minimizing GPU usage while maintaining high visual quality, we propose a novel real-time method that leverages the human visual system to adaptively adjust the resolution and frame rate of streamed rendered content under bandwidth constraints. By jointly considering image content, motion velocity, and network bandwidth, our method predicts the resolution and frame rate combination that delivers superior perceived quality while reducing rendering costs.

Abstract

Streaming rendered content is an attractive way to bring high-quality graphics to billions of mobile devices that do not have sufficient rendering power. Existing solutions render content on a server at a fixed frame rate, typically 30 or 60 frames per second, and reduce resolution when bandwidth is restricted. However, this strategy leads to suboptimal rendering quality under the bandwidth constraints. In this work, we exploit the spatio-temporal limits of the human visual system to improve perceived quality while reducing rendering costs by adaptively adjusting both frame rate and resolution based on scene content and motion. Our approach is codec-agnostic and requires only minimal modifications to existing rendering infrastructure. We propose a system in which a lightweight neural network predicts the optimal combination of frame rate and resolution for a given transmission bandwidth, content, and motion velocity. This prediction significantly enhances perceptual quality while minimizing computational cost under bandwidth constraints. The network is trained on a large dataset of rendered content labeled with a perceptual video quality metric.

Materials

• Paper:
  Streaming of rendered content with adaptive frame rate and resolution.
  Yaru Liu, Joseph March, Rafał K. Mantiuk.
  In SIGGRAPH 2026 Conference Proceedings
  [paper PDF]

Related projects

• ColorVideoVDP - A visual difference predictor for image, video and display distortions
• ASAP - Active Sampling for Pairwise Comparisons - used to efficiently collect AR-DAVID dataset
• pwcmp - Bayesian pairwise comparison scaling - used to scale AR-DAVID subjective responses