Probably I've written it not clearly enough: "Reason for GPL not allowing you to use code in your own programs, but allowing use through exec: you either release your code and contribute to open source or suffer from slower access methods."
Judging by both the source code and its placement in the repository (gen/FreeBSD/wordexp.c), I'm thinking it's a derivative of the FreeBSD implementation (which may itself be a derivative of the NetBSD implementation, or perhaps the other way around).
I'm guessing that the linked implementation probably hadn't been touched since 2008. My comment that this was source code from 2011 was referring to the fact that this release of libc (Libc-763.11) is from 2011.
wordexp, wordfree - perform word expansion like a posix-shell
So, the implementer decided to take the short route, and just spawn a shell which, essentially, gets passed the input data to wordexp(), to do the work. But, of course, having something that starts with such a comment...
/* XXX this is _not_ designed to be fast (...) wordexp is also rife with security "challenges", unless you pass it WRDE_NOCMD it must support subshell expansion, and even if you don't beause it has to support so much of the standard shell (all the odd little variable expansion options for example) it is hard to do without a subshell). It is probably just plan a Bad Idea to call in anything setuid, or executing remotely. */
...in your standard C library wasn't such a smart idea to start with. Scroll down, there are many more gems in the comments!
Sometimes it's better to just implement it as
void wordexp() {
fprintf(stderr,"wordexp() is a security nightmare. Not implemented.\n");
assert(0);
}
or decide to deliberately only implement a safe subset of the full functionality specified (e.g. only ~user-homedir expansion and $VARIABLES), to at least cover the common use-cases without creating a security nightmare.
"Like a POSIX shell" includes expanding command substitution. wordexp("`echo hi`") should return {"hi", NULL}. So it isn't totally ridiculous to reach for a shell; tracking multiple subprocesses, and nested command substitution (like "$(echo $(echo $(echo foo) $(echo bar)))") is a pain to reimplement correctly.
In an ideal world, we'd have a libsh that exposed all of the steps of what /bin/sh does in a nice fashion, this function would call one of the libsh functions, and /bin/sh would be a 10-line while (true) { libsh_this(); libsh_that(); }.
In a slightly less ideal world, the shell would have a way to separate things with null characters, without bothering Perl.
> In an ideal world, we'd have a libsh that exposed all of the steps of what /bin/sh does in a nice fashion, this function would call one of the libsh functions, and /bin/sh would be a 10-line while (true) { libsh_this(); libsh_that(); }.
The standard itself at http://pubs.opengroup.org/onlinepubs/9699919799/functions/wo... says "While wordexp() could be implemented entirely as a library routine, it is expected that most implementations run a shell in a subprocess to do the expansion" and gives guidance on how to implement it that way.
Is wordexp() a requiement for being POSIX compatible?
I searched through the FreeBSD repo on Gitbub and the only references to wordexp is basically the code itself. Searching the whole Github does not show many things either.
How common is the function in practice and how important is it?
common in practice? openbsd doesn't support it. afaik few programs use it, and those that do are usually easily patched not to (can use glob instead or disable whatever feature entirely).
Scroll down, there are many more gems in the comments!
...and what a spelling errors:
"This kludge is needed because /bin/sh seems to set IFS to the defualt even if you have set it; We also can't just ignore it because it is hard/unplesent to code around or even a potential security problem because the test suiete explicitly checks to make sure setting IFS 'works'"
ESR writes about sloppy spelling: "Write in clear, grammatical, correctly-spelled language. We've found by experience that people who are careless and sloppy writers are usually also careless and sloppy at thinking and coding (often enough to bet on, anyway)."
(Since the manual page strongly recommends not trusting wordexp with untrusted input even with WRDE_NOCMD, and based on a code search the function is rarely used in the first place, I don't think it's really sensitive.)
Performance is the least of my worries here. I'd be more worried about security, and plain dependency management.
I mean, let's say this libc thing is supposed to be installed in quite a lot of systems, are we really sure that perl thing is installed in all of those systems? And what if, by some crazy coincidence, perl happens to depend on libc?
In Debian, at least, there's actually a number of circular dependencies within the (small) set of packages marked as "essential"; they all sort of lean on one-another like a house of cards. Perl is in that set, if you're wondering.
> I mean, let's say this libc thing is supposed to be installed in quite a lot of systems
Apple's libc is not an independent thing to them - it's a part of OSX and iOS. You're not supposed to be able to take it out and use it for other things.
Apple libc in particular is supposed to be installed on exactly two base systems: OS X and iOS. The former has perl installed by default, and the latter marks wordexp as unavailable.
It's Apple's libc. They ship always ship Perl on OSX (and presumably iOS too?), so what do they care whether other platforms have to deal with an extra dependency? Their platform has Perl anyway, so the dependency is essentially free.
Sorry, I don't have a Mac to check, but does the OSX build of Perl depend on libc?
Also, that code seems to be from 2008 (or at least that's the latest year in the copyright header) despite the commit being from 2012. Does anyone know if this has been updated? Inserting a NUL byte between each word, and at the end doesn't sound like it requires Perl...
> Inserting a NUL byte between each word, and at the end doesn't sound like it requires Perl...
No, it's more clever than that. The shell will actually execute something like this:
/usr/bin/perl -e 'print join(chr(0), @ARGV), chr(0)' -- your input string goes here
So, what happens is that the shell (in this case, bash) performs argument expansion on your input string, then calls perl -e '...' with the expanded words as its arguments. What perl does is join all of those arguments with a NUL byte and spit them back out for the calling process to read, which makes figuring out where each expanded "word" begins and ends really simple.
Could you do this in shell? Probably. Would a correct implementation be as short and easy to understand? Probably not. Therefore, perl.
It would only be a problem if that specific perl command (perl -e 'print join(chr(0), @ARGV), chr(0)' -- args....) or the shell invoking it depended on wordexp(). Recursion is fine as long as it terminates.
With respect, is there evidence to support your claim?
My direct experience is that I have received value far in excess of what I have had to pay for the Apple products and services I have purchased over the years.
For example, I'm typing this reply on my PowerBook, which I use all day, every day in my work. I also have my iPhone with me constantly, and use it heavily.
Finally, the Apple brand was said to be worth about $104.7B vs. $62.8B for Microsoft in 2014. [1]
So, I am not saying you are wrong, only that I think some evidence would be helpful in making your case.
OT, but you're still using a Powerbook? That's pretty impressive. Which generation\year was it? I'm guessing you've had to swap out batteries and upgrade hard drive - have you had to modify anything else?
Here's the current implementation: http://opensource.apple.com/source/Libc/Libc-1044.1.2/gen/Fr...