Setting an arbitrary limit is now and will always be the wrong thing to do in API design. MAX_PATH is one of the most persistent reminders of this problem.
Neither 260 chars on Windows nor 1024 chars on the Mac is an actual limit of the filesystem. Both Windows NTFS/FAT32 and Mac HFS+ can actually support thousands of nested folders.
But the "MAX_PATH" concept persists and a large number of programs will fail mysteriously when the limit is reached. Even though the filesystem supports vastly more, we are stuck with operating systems that encourage programmers to needlessly limit things. And there's never been a serious push to improve the situation. Long paths aren't a serious alternative since they're not supported by many common Windows APIs (plus, users get confused when \\?\ appears in front of everything). It's completely insane.
The \\?\ thing is a weird hack. There are other side-effects of \\?\, for example the system will stop removing trailing spaces and dots from path elements. So you could have a filename that fits well within the limit of MAX_PATH but is inaccessible without \\?\ - just end it with a dot or a space.
It's not even always MAX_PATH directly that decides when you need \\?\. IIRC for a directory you must fit within MAX_PATH minus the length of a slash and an 8.3 filename.
There are other limits, for example by the time a create gets to the NT APIs the path must fit in a UNICODE_STRING structure which has a maximum length of 0xffff/sizeof(WCHAR). Once I was curious about probing the limits and I created an absolute path that was longer than this limit. I did it with something like this cmd script:
mkdir a
:top
ren a a.tmp
mkdir a
move a.tmp a\
goto :top
Sure enough I was able to create a dir that would stump just about anyone's recursive-directory-delete code. To delete it on my own filesystem I wrote a similar rename-loop to unravel it. If you ever want to play a prank on a Windows programmer who recursively enters directories, show them this.
I think you can blame C's poor string processing functions for that. Every time I have to deal with paths in C I prefer to allocate a static "MAX_PATH" buffer than dealing with dynamic memory allocation and string concatenation (did glibc finally include strlcat or will it take an other decade?).
Or better yet, I'd prefer to use a third party lib for that, but I don't know anything small and simple for the command line.
Neither 260 chars on Windows nor 1024 chars on the Mac is an actual limit of the filesystem. Both Windows NTFS/FAT32 and Mac HFS+ can actually support thousands of nested folders.
But the "MAX_PATH" concept persists and a large number of programs will fail mysteriously when the limit is reached. Even though the filesystem supports vastly more, we are stuck with operating systems that encourage programmers to needlessly limit things. And there's never been a serious push to improve the situation. Long paths aren't a serious alternative since they're not supported by many common Windows APIs (plus, users get confused when \\?\ appears in front of everything). It's completely insane.