I find Android a lot easier to use than iOS. What I've noticed when using an iOS device is that I often can't figure out how to do something. Then I research it and discover that it simply can't be done. Easy to learn, but unusable.
Android, on the other hand, trades discoverability for usability. I have heard that some Android users don't even know that they can pull down the notification bar. But once you learn to do that, it's a wonderfully usable system. (Same for long-press. What to do something with a thing you see? Press and hold. Works every time, but not something you would necessarily think to do.)
I think when people say "Android is unusable" they really mean, "Apple's icons are prettier", which is absolutely true. iOS devices look stunning on billboards and on TV. Android devices? Meh. (I blame the tastelessness of the carriers, though. A big fucking robot eyeball as the background? NASCAR and Sprint logos all over the place? Fuck off and die. AOSP is nice looking, though.)
(Want to see the biggest software disaster ever in the history of humanity? The Nook Color with Android. B&N's effort to cripple Android was so half-hearted that they unintentionally created the worst computing device ever made. Examples: you can select text and choose "copy", but they removed "paste" from the long-tap menu on text fields. You can copy, but not paste. They also made their own "buttons" that work like Android's; Back (which is an undiscoverable gesture, not a button), Menu, Home. But sometimes, you also get the normal Android buttons. So you have two menus (that work alike, but have different items), two back buttons (that work differently, sometimes), and so on. WHAT WERE THEY ON WHEN THEY DESIGNED THIS.
I'm going to be honest. I've never been so upset by anything ever. Using one of these things made me want to strangle the life out of everyone involved with the project and then go buy an iPad.
And oh yeah, the fucking screen flickers like a dying fluorescent light. HOW CAN YOU FAIL SO BADLY.
The one good thing is that they do not advertise that they are using Android.
I've been using vanilla Android for almost a year now. The more I use it the more I dislike it. By far my number one complaint is bugs. Some of them are so apparent and easily reproducible is simply inexcusable that they're still there (all reported btw). I haven't rooted my Nexus One, so I don't have that to blame either.
It gets tiring to click the same message-contact shortcut and sometimes to be taken to the right place and sometimes to the list of users, or trying to attach an image and not be sure if the gallery application will crash or not (and having to manually close it), or after N text messages sent and getting a lovely "Message could not be sent" error not knowing if it's the messaging app misbehaving or my carrier (almost always a force close of the messaging application fixes the issue), or the scrolling in the browser acting up while locking and unlocking the screen miraculously fixes it... There's also the focus on fancy features like navigation while neglecting the basics like the phone part. I had more control / features on the phone side with my ancient Sony Ericsson dumb phone than on my "superphone"
I think you're just talking about the applications in a stock Nexus One, not the operating system itself (i.e. the UI, the services it provides, etc). If the apps are lousy it's one thing, but the system behaves quite predictably more often than not.
And yes, the stock apps could do with a bit improvement, but you can always install another alternative app from the market. That's what it is for, isn't it?
I didn't (and wouldn't) say Android is badly designed. It's OK, and I'd even say certain parts are beautiful, but it's quite uneven and some parts do actually border on ugly. It is certainly good enough for me, but for most people I think the bar has been raised to stratospheric levels by Apple and people now have really high expectations of the visual experience they should get from phone OSes.
I don't know about you, but Android 2.2 is pretty beautiful. I had an iPhone for a while and personally feel the design of Android is significantly more natural to me. Best of all if you don't like how something behaves, you can usually swap it out.