Metal likes to be formed by rolling, and it's relatively easy to image how cans are cut to length, passed through a die and seam welded. Obviously that's a lot of capital for one can size so round dies are easily swapped for various size cans on the same line. Forming stamped and bent edges would be much more intensive for changeovers.
Lids would require directional placement:
Round shapes fit in all directions, pretty clear here that any other shape would require it to be directionally correct.
Can liners are sprayed:
The inside of your cans are coated to protect the food, corners are harder to maintain an even sprayed coating
There are plenty of others I'm sure and this is a rambling post but maybe it gives a little more insight into the world of cans.
However, part of me thinks that given some time and ingenuity, we'd come up with techniques that are better suited for the efficient production of cuboids. In other words, most of the things you mention are conceivably solvable by the right tooling (i.e. fixed costs). If that were the case, I'd have a hard time believing that marginal/per-unit costs would be significantly higher, and I think it would be interesting to look at the savings in shelf and transportation space compared to any of those increased production costs.
I would guess that cardboard is less expensive to source and work, so they can afford to take the extra effort to make it a box.
I think the bottom of a cylindrical can might be doable but rolling the lid seam on a filled can would be tough.
That salt will impart flavour on the food.
So you can argue it both ways. Even just a little bit of a metallic taste in most food is quite unpleasant.