DiagSplat · controlled RAW-noise experiment · 2 August 2026

RawNeRF synthetic noise:
the target failure appears, but NTF v0 does not optimize

Vanilla 3DGS and a first noise-adaptive transfer-function prototype on RawNeRF / sharpshadow. This page separates two findings: noisy RAW does drive Gaussian proliferation and extreme anisotropy, while the current NTF implementation falls into a zero-gradient trap at black initialization.

Training & test data ↗31 noisy RAW train views5 clean RAW test viewscalibrated Bayer noise ×641008 × 75630k iterationsNTF v0 · invalid optimization

Clean held-out target, Vanilla reconstruction, and black frozen NTF reconstruction

Vanilla clean-test PSNR26.11 dBdisplay-space PSNR 21.66 dB

Vanilla Gaussian count185,47212.0× the 15,505 initialization

Vanilla aspect ratio > 1095.6%79.9% are above 100

NTF clean-test PSNR12.16 dBblack render · optimization invalid

结论：这是一次有信息量的失败实验

关于问题本身：Vanilla 从 15,505 增长到 185,472 个 Gaussian，并形成大量极端细长 Gaussian，和 Raw3DGS 对 noisy RAW 的诊断一致。这个小场景已经能触发我们关心的 failure mode。

关于当前方法：NTF v0 的 96.5% “点数减少”不能视为成功；它输出全黑，在 7k 与 30k 的参数逐字节相同。当前结果只证明实现的零点梯度设计不适用于 3DGS 的黑色初始化。

Do not report NTF −96.5% Gaussians as a winNTF 的 6,477 个点仍保持统一黑色 DC coefficient、统一 0.01 opacity 和各向同性初始尺度。它不是更紧凑的重建，而是被 opacity reset/pruning 清空后的未训练模型。

训练数据和测试数据长什么样

整个场景有 36 个已配准视角。我们按照固定顺序每隔 8 张抽一张作为测试视角；其余视角只把合成 noisy RAW送入 3DGS。测试视角从未参与优化，并始终使用clean RAW作为评价目标。

36 views同一场景、同一相机轨迹

→

31 trainsynthetic noisy RAW

5 testclean held-out RAW

TRAIN · 31 VIEWS

优化时真正看到的 noisy RAW

下列图像是统一显示 ISP 后的预览。模型实际读取 camera-linear RGB；噪声在 Bayer RAW 域注入，clean counterpart 仅用于审计数据生成，不会进入训练 loss。

Noisy RAW training frame IMG 7121 — IMG_7121 · noisy train

Noisy RAW training frame IMG 7135 — IMG_7135 · noisy train

Noisy RAW training frame IMG 7155 — IMG_7155 · noisy train

TEST · 5 VIEWS

只用于评价的 clean held-out RAW

这些相机视角不参与训练，也不添加 synthetic noise。最终的 PSNR、SSIM 和 LPIPS 都把模型的新视角渲染与这些 clean 图像比较。

Clean held-out test frame IMG 7120 — IMG_7120 · clean test

Clean held-out test frame IMG 7136 — IMG_7136 · clean test

Clean held-out test frame IMG 7152 — IMG_7152 · clean test

Subset	Count	Image used by model/evaluator	Clean paired image
Train	31	Synthetic noisy RAW · participates in optimization	Stored for protocol inspection only; not used by loss
Test	5	Clean RAW · evaluation only	It is the test ground truth

三组完整 clean/noisy/noise-map 配对在页面后面的 “Synthetic RAW-noise protocol” 中展开；五张测试图及其 Vanilla/NTF 渲染也在后文逐张展示。

定量结果

Method	Linear PSNR ↑	Display PSNR ↑	Linear SSIM ↑	LPIPS ↓	Gaussians	Status
Vanilla 3DGS · 30k	26.1102	21.6608	0.8387	0.2567	185,472	valid
NTF-GS v0 · SNR 10 dB	12.1573	7.3857	0.2610	0.4755	6,477	frozen / invalid

Linear metrics use the standard Graphdeco evaluator on five clean held-out camera-linear RAW views. Display PSNR is recomputed after the same cam2rgb, clipping and sRGB mapping used below; Vanilla display-space SSIM is 0.5343, showing that the ISP makes residual color/high-frequency errors more visible than the linear PSNR suggests.

PSNR trajectory, Gaussian count trajectory, and anisotropy statistics

Why NTF v0 freezes

1 · Black sparse colorsRawNeRF COLMAP points contain RGB = (0,0,0), so all initial DC coefficients encode black.

2 · Initial render is zeroThe first 3DGS render is black or below the transfer floor.

3 · Hard clampclamp(render, x_min) makes the transfer value finite but its derivative zero below the floor.

4 · No color gradientDC coefficients remain exactly −1.7724538; the clean-test PSNR stays exactly 12.377 dB in the in-training evaluator.

5 · Reset then pruneOpacity becomes uniformly 0.01, points fall from 15,505 to 6,477, and the model remains black through 30k.

T(x) = piecewise_log( log₂(clamp(x, x_min) / x_min) ) For x < x_min: d clamp(x, x_min) / dx = 0 Therefore at a black initialization: dL / d(render) = 0

This also explains why the earlier RealX3D run did not collapse: its initialization/rendering path had enough nonzero color support to escape the clamp floor. The RAW setup exposes a previously hidden optimization boundary condition.

What Vanilla tells us about noisy RAW

Positive mechanism signal: Vanilla recovers the scene structure, but the display-corrected views remain visibly blurred and color-fringed while its representation grows aggressively: 118,704 Gaussians at 7k and 185,472 at 30k. At 30k, 95.6% have scale aspect ratio above 10 and 79.9% above 100; the median ratio is 1,526.

This strongly matches the “thin / flat Gaussians absorb noise” hypothesis. However, it is not yet a complete causal measurement of excess growth because the clean-RAW Vanilla control has not been run. The next report should compare noisy Vanilla against clean Vanilla under identical cameras and schedule.

Five clean held-out views

Click any image to inspect it at full resolution. Error maps show mean absolute camera-linear RGB error, clipped at 0.12 and rendered with an inferno color scale.

Held-out view 1

IMG_7120.JPG

Clean held-out view IMG 7120 — Clean RAW target · display corrected

Vanilla render IMG 7120 — Vanilla 3DGS · 30k

Black frozen NTF render IMG 7120 — NTF v0 · frozen black

Vanilla error map IMG 7120 — |Vanilla − clean|

Held-out view 2

IMG_7128.JPG

Clean held-out view IMG 7128 — Clean RAW target

Black NTF render IMG 7128 — NTF v0 · frozen

Vanilla error map IMG 7128 — |Vanilla − clean|

Held-out view 3

IMG_7136.JPG

Clean held-out view IMG 7136 — Clean RAW target

Black NTF render IMG 7136 — NTF v0 · frozen

Vanilla error map IMG 7136 — |Vanilla − clean|

Held-out view 4

IMG_7144.JPG

Clean held-out view IMG 7144 — Clean RAW target

Black NTF render IMG 7144 — NTF v0 · frozen

Vanilla error map IMG 7144 — |Vanilla − clean|

Held-out view 5

IMG_7152.JPG

Clean held-out view IMG 7152 — Clean RAW target

Black NTF render IMG 7152 — NTF v0 · frozen

Vanilla error map IMG 7152 — |Vanilla − clean|

Synthetic RAW-noise protocol

Noise is sampled before demosaic in normalized Bayer space using the iPhone X DNG NoiseProfile, multiplied by 64 for a strong first stress test. Clean and noisy variants share demosaic, downsampling, lens undistortion, quantization and display processing.

Training pair A

IMG_7121.JPG

Clean RAW training reference IMG 7121 — Clean daylight RAW

Synthetic noisy RAW IMG 7121 — Noisy training target

Noise magnitude IMG 7121 — |noisy − clean| · inferno

Training pair B

IMG_7135.JPG

Clean RAW training reference IMG 7135 — Clean daylight RAW

Synthetic noisy RAW IMG 7135 — Noisy training target

Noise magnitude IMG 7135 — |noisy − clean| · inferno

Training pair C

IMG_7155.JPG

Clean RAW training reference IMG 7155 — Clean daylight RAW

Synthetic noisy RAW IMG 7155 — Noisy training target

Noise magnitude IMG 7155 — |noisy − clean| · inferno

Minimum next experiment

1 · Make the toe differentiableReplace the hard-floor log with log1p(x / τ) or asinh(x / τ). Both preserve a finite, nonzero derivative at x = 0. A straight-through clamp is useful only as a diagnostic ablation.

2 · Add the clean controlRun Vanilla on the exact clean training views. The difference between clean-Vanilla and noisy-Vanilla is the causal measurement of noise-induced Gaussian growth.

3 · Rerun noisy NTF v1Require early evidence before 30k: non-black DC coefficients, rising test PSNR by 1k, and Gaussian population above zero-reset collapse. Stop early if any fails.

Required three-cell comparison

A · Clean RAW + Vanilla
Missing control; measures normal scene complexity.

B · Noisy RAW + Vanilla
Done; 185,472 points and extreme anisotropy.

C · Noisy RAW + NTF v1
v0 failed; rerun after differentiable-toe fix.

Suggested diagnostic ablations: projected-image color initialization and a 1k Vanilla warm start can verify the zero-gradient diagnosis, but neither should replace the main differentiable-transfer solution.

Reproducibility notes

Dataset	RawNeRF / sharpshadow · 36 iPhone X DNGs
Split	31 noisy train · 5 clean held-out
Noise	variance = 64 × (3.28249e−5 × signal + 4.34317e−7), sampled in Bayer space
Geometry	Official COLMAP poses; OPENCV distortion converted consistently to 1008 × 756 PINHOLE
NTF effective luma model	shot = 4.99490e−4 · read std = 1.06341e−2 · target SNR = 10 dB
Artifacts	dataset manifest · Vanilla metrics · NTF metrics

Related primary sources: RawNeRF · From Chaos to Clarity: 3DGS in the Dark.