Seems a lot of people feel the way you do. I had to make something using swift once, and could barely find a community to help me decipher the cryptic swift documentation centered around their coreml library. Apple is keeping Swift on life support, imo.
CoreML is a bit of an exceptional challenge beyond just Swift. Admittedly it could really use a lot more documentation from Apple, but it would be unfair to judge swift based on an uncommon and under documentated library (I suspect that it would get a lot more use if it were not so hard to set up).