PayPal is _absurdly_ easy to set up and start collecting money.
You do though, need to understand that Papal's main business advantage over banks is their automated fraud prevention/detection - if you do anything that might come close to triggering their automated systems, you need to watch out. In my experience, that's almost always been accepting payments significantly before shipping something that can be tracked. Pre orders often get accounts locked. Non physically shipped products (thing "digital downloads" - software, music, or ebooks) often get accounts locked.
If you're not doing that kind of thing, or if you're doing it and are prepared to put up with Paypal holding on to your money until they believe your customers are happy, then PayPal is a reasonable solution (we have a client who starts accepting pre-orders for e new edition of his book every October, and PayPal hang on to _all_ money put into his PayPal account until 3 or 4 weeks after the books start shipping in early January. This has been going on for 4 years now, exactly the same every year, but the client is perfectly happy with that situation, he's mainly accepting pre-orders as a way to more accurately guage how many copies to print in the first run, he doesn't need to have the money up front but it's a great way for him to know really firm minimum order numbers for the first print run. It's only when he gets drunk over Xmas/new year and starts ringing PayPal up and screaming "give me my fucking money!" at anyone who answers the phone that it's a drama, unfortunately _that's_ been going on for 4 years now too...
Tl;dr, use PayPal to start with. Never leave more money in the PayPal account than you're prepared to lose. Never leave that money in a bank account linked to PayPal either. Make sure you've got a backup payment method at least planned, that you can switch in sufficiently quickly that you don't go broke between when PayPal cuts off your money and the new payment system is in place (that means get your merchant account and Internet merchant ID applications in process _now_, not after you have a PayPal problem.)