Also, Adam Smith's example of the invisible hand was as a mechanism to prevent so-called free trade, and raise protectionist tariffs. The people you mentioned have twisted Smith's words into the exact opposite meaning - they say Smith's invisible hand sweeps away protectionist tariffs and allows international free trade. He said the complete opposite of what they say he said.
Thanks, I was going to mention the context of use but needed to go back to confirm what the usage was.
I am aware that his one use of "free market" was in a passage describing protectionist trade practices favouring the woolens manufacture industry in England: keeping raw wool import costs low and preventing import of finished goods, thereby maximising the revenue-cost differential, which is to say, profits.
Here's the beginning of that section, which generally inveighs against protectionist retraints on trade, most especially the "Corn Laws" -- limitations on grain imports to England:
By restraining, either by high duties or by absolute prohibitions, the importation of such goods from foreign countries as can be produced at home, the monopoly of the home market is more or less secured to the domestic industry employed in producing them. Thus the prohibition of importing either live cattle or salt provisions from foreign countries secures to the graziers of Great Britain the monopoly of the home market for butcher's meat. The high duties upon the importation of corn, which in times of moderate plenty amount to a prohibition, give a like advantage to the growers of that commodity. The prohibition of the importation of foreign woollens is equally favourable to the woollen manufacturers. The silk manufacture, though altogether employed upon foreign materials, has lately obtained the same advantage. The linen manufacture has not yet obtained it, but is making great strides towards it. Many other sorts of manufacturers have, in the same manner, obtained in Great Britain, either altogether or very nearly, a monopoly against their countrymen. The variety of goods of which the importation into Great Britain is prohibited, either absolutely, or under certain circumstances, greatly exceeds what can easily be suspected by those who are not well acquainted with the laws of the customs.
That this monopoly of the home market frequently gives great encouragement to that particular species of industry which enjoys it, and frequently turns towards that employment a greater share of both the labour and stock of the society than would otherwise have gone to it, cannot be doubted. But whether it tends either to increase the general industry of the society, or to give it the most advantageous direction, is not, perhaps, altogether so evident.
The restraint of government intervention (here as so often elsewhere noted in Wealth) is by government, yes, but quite clearly on behalf of specific powerful commercial interests. It's that* power Smith is hoping to curb -- directly then a restraint on excessive power accumulation by means of commerce, trade, and manufacture.