Scale during training

[rwn17]

Hi @botaoye ,

Thanks for your excellent work!

I noticed that during training, there doesn't seem to be any alignment between the Dust3r-predicted Gaussian and the GT pose. However, since Dust3r is supposed to predict scale-invariant depth, I didn't observe any alignment between them. I wondered if I missed something or if the network learned the metric scale during the training process. Thanks in advance!

Bests,
Weining

[botaoye]

Hi, NoPoSplat does not rely on the output of DUSt3T or MASt3R, but uses a similar canonical space prediction methodology used in DUSt3R to train the whole model, so we don't need to do an alignment between DUSt3R.

Our method cannot actually predict metric-scale Gaussians either, since the training data does not have a ground truth pose for the metric scale. However, our method can predict Gaussians that match the intrinsics of the input images, as detailed in this paper. We also employ a pose normalization to solve the scale ambiguity problem during training.

[rwn17]

Thanks for your explanation!