The default behaviour of PowerShell is to stream objects one by one, the same as typical UNIX shells. In principle, it can process unlimited amounts of data on a single pipeline with a small, fixed amount of memory. The exceptions are commands like Sort-Object, which do require everything to be held at once in RAM. In theory, it could do an offline sort like the UNIX "sort" command does, but the issue is that that might break some scripts that rely on .NET objects that aren't serializable. If you're super keen, it would be possible to add this feature and develop a "Sort-ObjectOffline", at the risk that very rarely it might shred some objects...
The problem with JSON is that it does not support streaming by default. It's possible to use non-standard JSON-like formats to work around this, but then you're no longer using JSON!
"The probelm with JSON is that it does not support streaming by default."
This summarises the problem I have with JSON more succintly. It was not designed for streaming, thus "it does not support streaming by default".
Non-standard, line-oriented JSON formats are usable, although as a user I cannot see how they offer any significant improvement over previous approaches with fewer brackets, braces, colons, commas and quotes (BBCCQ). Consider the BSD utility mtree or the BSD-version of stat. These have options to output text in "shell-friendly" formats,1 minus all the BBCC and excessive Q. Sure, people could add options to utilities to output XML, or line-oriented JSON, but generally they don't. Why is that. Perhaps there is a reason.
You said it best: "It's possible use non-standard JSON-like formats to work around [JSON's limitations], but then you're no longer using JSON!"
Maybe JSON is just about hype or something. An attractant for today's "developers". This would explain why I am just not attracted by it.
There were some upvotes then a series of downvotes as the top comment in this thread changed. Thus, opinion on this issue is mixed.
JSON of course stands for "object" notation. Once we start delimiting the "object" with a newline, or limit its size/length, it arguably starts to sound more like string and the problem of memory requirements is abated.
As the parent comment suggested, when the "object" notation is used for delimited JSON it's really not "JSON" anymore. Is it still describing "objects", or is it describing strings, with the addition of brackets, braces, commas, colons and more quotes. Let the reader decide.
It is reasonable to ask what was the purpose of JSON in the first place, and, if it was designed for sending data over a network, whether previous solutions like netstrings could accomplish the same things as delimited JSON.
The problem with JSON is that it does not support streaming by default. It's possible to use non-standard JSON-like formats to work around this, but then you're no longer using JSON!