Focal-aware ph_opt
DragonBunny · matched 1K hard-binary evaluation

Focal-aware multiplane reconstruction

Reference, the original all-plane baseline, and the selected focal-aware method under the same initialization, capacity and 1,000-iteration budget.

1,000 iterations9 focal planes8 × RGB SLM patternsHard binary outputs3 × 5 s unseen trajectories
Method

Optimize the 3D stack and its refocusing dynamics

Focal-Aware Reconstruction

Objective

Four smooth 17-frame training trajectories sample continuous focal states. They are independent of the three held-out trajectories below.

Continuous focal render\[\widehat{F}_t=\operatorname{sRGB}\!\left(\sqrt{(1-\alpha_t)\widehat{A}_{\lfloor d_t\rfloor}^{,2}+\alpha_t\widehat{A}_{\lceil d_t\rceil}^{,2}}\right)\]
Full loss\[\mathcal{L}=\mathcal{L}_{\text{9-plane}}+\mathcal{L}_{\text{gaze}}+\lambda\!\left(\mathcal{L}_{\Delta}+0.1\,\mathcal{L}_{\text{CSF}}\right)\]
Focal-video difference loss\[\mathcal{L}_{\Delta}=\rho_{\epsilon}\!\left(\Delta\widehat{F}-\Delta F^{\star}\right)+0.25\,\rho_{\epsilon}\!\left(\Delta^{2}\widehat{F}-\Delta^{2}F^{\star}\right)\]
Gradient-balanced auxiliary weight\[\lambda=\rho\,\frac{\left\lVert\nabla_{\theta}(\mathcal{L}_{\text{9-plane}}+\mathcal{L}_{\text{gaze}})\right\rVert_{2}}{\left\lVert\nabla_{\theta}\mathcal{L}_{\text{aux}}\right\rVert_{2}}\]
Iterations 1–800: soft Gumbel, \(\rho=0.10\).   Iterations 801–1000: hard-forward STE, \(\rho=12\).
Implementation branch ↗

Why it works

1
Continuous focal samples expose interpolation artifacts.

Nine discrete targets cannot see errors that appear only between planes.

2
First- and second-order differences suppress refocus flicker.

The loss matches focal motion and acceleration instead of applying CVVDP directly as a training loss.

3
Saliency-weighted opponent CSF spends capacity where artifacts are visible.

It emphasizes perceptually important spatial and temporal residuals.

4
Plane/gaze anchors and hard-forward STE protect deployment quality.

The nine-plane stack remains stable while the final 200 iterations optimize exact binary forward patterns.

Mean novel CVVDP
8.079 JOD
+0.858
Mean novel saliency PSNR
26.656 dB
+1.545 dB
Mean PSNR across 9 planes
25.387 dB
−0.0004 dB
Results

Matched hard-binary metrics

Novel-video metrics average three unseen 125-frame trajectories; plane metrics average nine discrete focal planes.

Metric1K baselineFocal-AwareChange
Mean novel-trajectory CVVDP JOD ↑7.22098.0793+0.8584
Mean novel saliency PSNR ↑25.1111 dB26.6563 dB+1.5452 dB
Mean novel saliency SSIM ↑0.570850.67023+0.09938
Mean novel full-frame PSNR ↑26.0652 dB26.4263 dB+0.3611 dB
Mean temporal-delta residual ↓0.0014940.001410−5.59%
Mean PSNR — 9 planes ↑25.3873 dB25.3869 dB−0.0004 dB
Mean SSIM — 9 planes ↑0.517010.52452+0.00751
Mean LPIPS — 9 planes ↓0.507580.50061−0.00697
Worst-plane PSNR ↑24.8259 dB24.4988 dB−0.3271 dB
Visual comparison

Three unseen focal trajectories

Each video is 5 seconds at 25 fps. The red circle guides gaze; the upper-right plot shows the continuous focal coordinate.

1280 × 720 web videos

A · Predicted RGB saliency

Center-biased scan across salient Dragon and Bunny regions.

+0.771 JOD · +1.230 dB
Focus curve for predicted RGB saliency trajectory

Reference

Continuous RGB-D focal render

1K Baseline

7.216 JOD · 25.022 dB saliency PSNR

Focal-Aware

7.987 JOD · 26.252 dB saliency PSNR

Representative low-focus state — Reference | Baseline | Focal-Aware

Open full-resolution comparison
Reference, baseline and focal-aware low-focus comparison

B · RGB-D near-object bias

Depth-weighted saliency emphasizes near Dragon/Bunny content.

+0.903 JOD · +1.686 dB
Focus curve for RGB-D near-object trajectory

Reference

Continuous RGB-D focal render

1K Baseline

7.206 JOD · 25.313 dB saliency PSNR

Focal-Aware

8.109 JOD · 26.999 dB saliency PSNR

Representative mid-focus state — Reference | Baseline | Focal-Aware

Open full-resolution comparison
Reference, baseline and focal-aware mid-focus comparison

C · RGB-D boundary stress

Depth-gradient weighting crosses foreground/background boundaries.

+0.902 JOD · +1.720 dB
Focus curve for RGB-D boundary trajectory

Reference

Continuous RGB-D focal render

1K Baseline

7.240 JOD · 24.998 dB saliency PSNR

Focal-Aware

8.142 JOD · 26.718 dB saliency PSNR

Representative high-focus state — Reference | Baseline | Focal-Aware

Open full-resolution comparison
Reference, baseline and focal-aware high-focus comparison