+1 on smolvm! I provision a smolvm per job in preloop, a local/self-hosted github actions( https://github.com/preloopdev/preloop).
iirc there was also some work to add a deny-cidr option as well which would give you more flexibility. But the egress filter captures most of what you need so it works great nonetheless.