Photogrammetry struggles with certain types of materials (e.g. reflective surfaces). It's also very difficult to capture fine details (thin structures, hair). 3DGS is very good at that. And people are working on improving current shortcomings, including methods to extract meshes that we could use in traditional graphics pipelines.
3DGS is absolutely not good with non Lambertian materials..
After testing it, if fails in very basic cases. And it is normal that it fails, non Lambertian materials are not reconstructed correctly with SfM methods.
You use SfM to find the first point cloud. However SfM is based on the hypothesis that the same point 'moves' linearly in between any two views. This hypothesis is important because it allows you to match a point in two pictures, and given the distance between the two images, you can triangulate the point in space. Therefore find it's depth.
However, non-Lambertian points move non linearly in viewing space (eg a specular point depends on the viewer pose).
So, automatically, their positions in space will be false, and you'll have floating points.
Gaussian 'splats' may have the potential to render non-Lambertian stuff using for example the spherical harmonics (even if I don't think the viewer use them if I'm not mistaken). But, capturing non-Lambertian points is very difficult and an open research problem.
