Ah I see, that makes more sense. If you had two NVIDIA GPUs while one of them was passed through to a guest then I think that problem may still exist.
However if you only have one nvidia gpu (and have a different kind of gpu for display), then I think simply running "sudo rmmod vfio" "sudo modprobe nvidia" would work.
However if you only have one nvidia gpu (and have a different kind of gpu for display), then I think simply running "sudo rmmod vfio" "sudo modprobe nvidia" would work.