Yes, the court can make such claims as they do in Powell, and Congress can ignore them. It is pure power politics, but the reputation of the court has been so diminished that Congress could easily get away with refusing to seat a member or two and there is really nothing that anyone could do to stop them.
You can make that argument about anything, though and it be would be equally applicable. 'The branches of the US government can start ignoring each other'. Sure, they could but that possibility doesn't really offer any useful insight. It's like saying I can become world heavyweight boxing champion by shooting my opponents.
https://en.wikipedia.org/wiki/Powell_v._McCormack