I’m starting to suspect something is wrong with our Elgato card because it has been a hit and miss experience for a long time now. We also have a knock off chinese 4k stream box (not the total dirt cheap ones) and that has sometimes been better than the elgato.
You need to make sure that you're not connecting it to an overloaded USB hub and that it's connected to a USB 3.0 port. I get framerate drops when those conditions aren't satisfied.