If you want to start with CUDA, you can buy EC2 instances with GPUs and pay for time used.
If you want to start with deep learning, you don't need GPUs or CUDA. All the popular frameworks work fine on CPUs. GPUs are not guaranteed to accelerate all deep learning use cases; sometimes the time taken to transfer data to and from GPU dominates the time taken to process that data.
If you want to start with deep learning using an AMD GPU, Theano has support for OpenCL which is an alternative to CUDA. But as I understand, this support still remains limited and incomplete.
If you want to start with CUDA, you can buy EC2 instances with GPUs and pay for time used.
If you want to start with deep learning, you don't need GPUs or CUDA. All the popular frameworks work fine on CPUs. GPUs are not guaranteed to accelerate all deep learning use cases; sometimes the time taken to transfer data to and from GPU dominates the time taken to process that data.
If you want to start with deep learning using an AMD GPU, Theano has support for OpenCL which is an alternative to CUDA. But as I understand, this support still remains limited and incomplete.