In the vast majority* of cases, agencies of the US government should have all non-classified data OCRed, indexed, hosted on their site and additionally available on BitTorrent. Put together a reasonable "Open Access Data" budget and make it available under the Library of Congress.
* Thinking things like astronomical data or other petabyte+ data sets.
Indeed, I couldn't agree more. Though I'd be happy for them to be coerced into providing the petabyte+ datasets too. If they can afford to store them they should serve them.
I believe the UK signed up along with the US as part of the G8 to an Open Data Charter [1] in 2013. In theory it aims to ensure that all government data be published openly by default, but in practice, as far as I can work out it has no teeth. You can point government departments to it, tell them they should release data, ask why they wouldn't and just get stonewalled.
I particularly pushed hard to try and get data from speed cameras released (without personal details; just time, location, speed, etc.) I even offered to aggregate data, normalise it, run the hosting, etc all at my own cost. Nothing. As far as I can tell there's no one to go to who has the power and influence to make it happen.
* Thinking things like astronomical data or other petabyte+ data sets.