I'd recommend to start by using a platform for first experiments, like fine-tuning offerings, which can be done for lower cost and will handle the thornier issues therein.
Do some experiments to see if fine-tuning shows potential for improvements, curating the dataset and reward function are the harder parts anyway.
the other thing is that the little models are getting better and more capable, so the hardware ROI becomes greater over time