You could use a real car too! (As long as low speed / open lot), I just did an experiment like this last weekend, a similar stack (build off of Openpilot/Comma 4) could work with the fly model, will email you some details if you're curious
True RC car is definitely the next logical step :) Yeah as long as you set a low enough speed limit, do it in an empty parking lot, have a human always ready to brake, I think it should be relatively safe haha