Whatever plastic particles you're eating in fish flesh really are a drop in the bucket. You'd get a lot more benefit by throwing away all your clothing, carpets, bedding, etc that contain synthetic fibers and replacing them all with pure cotton, wool, silk or flax but I doubt you're in a rush to do that. Unless you're willing to go that far, nothing else you do will make a difference.
So are you, and everybody else that has synthetic textiles in their house.