Hacker News new | past | comments | ask | show | jobs | submit login
Microsoft_craziness.h (2018) (gist.github.com)
200 points by lpcvoid 13 days ago | hide | past | favorite | 118 comments





Something I always wondered about Visual Studio and Windows programming:

If you compile a C++ program with Visual Studio, it gets dynamic dependencies on some visual studio DLLs. DLLs that are not necessarily installed by default on the Windows of the target user you're compiling for.

So the question is: MS created both Windows and Visual Studio. So why can't Visual Studio compile to something that works by default on the Windows of the target audience without requiring them to install those additional DLLs? Doesn't the win32 API provide enough, and can't Visual Studio statically link in whatever extras it needs, or those extras be added to the Windows API that is present by default if they're that necessary?

Also, given that Windows comes with pre-installed programs that were written in C++, how did those programs get compiled to not require those Visual Studio dependencies, are MS not using Visual Studio themselves internally?


I mean, Visual Studio does this (compiles against its own version of the C/C++ standard library) to avoid the classic nonsense that every Linux user sees when they build a binary and bring it to an older OS then it demands a newer version of glibc and segfaults - this is why every commercial Linux app has to compile on the oldest Linux they can possibly find, and it's frustrating.

You can either:

* Statically link, so you don't need these DLLs at all

* Just copy the DLLs along with your app in the same folder

* Ask users to install the VC Redist package

Option #2 is typically the best one and is pretty easy to do. macOS typically does #2 implicitly (i.e. libswift.framework et al), but ships all of those binaries in a "bundle" that is hidden from users.


> macOS typically does #2 implicitly

On macOS libSystem.dylib provides the C runtime (i.e. libc). It cannot be statically linked, and AFAIU you cannot ship it with your own project.[1] To support older environments you declare an ABI/API target using MACOSX_DEPLOYMENT_TARGET, -miphoneos-version-min, or similar toolchain option. It's normative for projects targeting an Apple platform to explicitly declare a target several releases in the past, but without an explicit declaration the default is to build a binary targeting the version of the SDK you're using, which would normally be the most recently released.

Bundling your own dynamic libraries is trivial on Linux by simply defining the run-time search path (aka rpath) when building your application. In general, the systems's dynamic linker is entirely agnostic. And this is even easier on Linux than most other platforms as you can use the $ORIGIN macro to define paths relative to the main binary rather than using fixed absolute paths. $ORIGIN was supported several years before the similar capability on macOS, @loader_path and @executable_path.

Bundling dynamic libraries is extremely simple on Linux, it's just not common.

Linking to an older glibc ABI is also possible--glibc rigorously utilizes ELF symbol versioning--but admittedly very difficult. None of the build toolchains provide built-in facilities (i.e. not even anything similar to the musl-gcc wrapper for cross-compiling against musl on glibc-based systems), and for the most part glibc only versions its run-time binary symbols, not its compile-time header API. That means you basically need to download old glibc headers or, as most people tend to do, build on old distributions.

OTOH, on Linux you can at least statically link your own libc on account of the kernel's [relatively] stable ABI.

In practice neither Linux, macOS, nor Windows provide an ideal situation. But at least in principle Linux supports all the common permutations anyone may want, excepting the absence of any singularly blessed system libc. (But to reiterate both musl libc and especially glibc provide strong backwards ABI compatibility, notwithstanding the absence of compile-time knobs for selecting older ABIs.)

[1] Perhaps you can hack things to load a bundled libSystem copy. I have a vague memory of someone pointing that out on HN before. But in any event that's definitely not a supported feature.


"In principle it should work, but in reality it doesn't", boy isn't that the Linux story in a nutshell :)

> Option #2 is typically the best one and is pretty easy to do

Your app's manifest is more important than you realise and I wonder if the OP is aware of this? Its SxS not SxSW. :-)

http://tinyurl.com/y3q8h3ks

This might be useful as well https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-...

IIRC Windows has been UTF-16 for ages which this link suggests has been since 2000. https://en.wikipedia.org/wiki/Unicode_in_Microsoft_Windows

I always treat UTF-8 as internet territory like web pages because its reportedly easier to recover from corruption which occurs more often online than with the components inside a computer.


> * Just copy the DLLs along with your app in the same folder

Gosh I wish things worked like this by default on Linux (dynamic linker search path by default starts with the directory where the binary is located)


For people who don't know, it's common to launch your Linux app through a proxy to do just this. Super easy to do.

>Just copy the DLLs along with your app in the same folder

I think the license precludes distributing the DLL's with your application. You are supposed to use the VC Redist package.


No you can redistribute them, that's why they are called redistributable. See here: https://docs.microsoft.com/en-us/cpp/windows/determining-whi...

Not the Debug versions though.

Why would you ship an application with the debug versions of the DLLs though?

The redist installer is recommended but not required.

I'm not sure it's even recommended any more. It's technically better because it means that Microsoft can apply security patches out-of-band, but on the other hand, it means that Microsoft can change The Code Your App Executes without you testing it, and tbh if you're writing a production app you should probably make sure your libraries are up-to-date anyways

> compiles against its own version of the C/C++ standard library

The C++ library yes. However IIRC Windows now ships a conforming somehow-modern C standard library (ucrt) and modern MSVCs link to it. It is probably also included in current redistributables for the still supported (by MSVC) old versions of Windows that did not have it. Also, the forever binary incompatible C++ standard lib had its limit regardless of the platform (more because of 3rd party binary libs) and MS has switched to a backward binary compatible C++ redistributable for a few years now (they may rebreak in the future).

So on some points (but not all, and it is likely to never be all) the situation is converging.

> to avoid the classic nonsense that every Linux user sees when they build a binary and bring it to an older OS

The thing with building a binary-only program for "Linux" is that "Linux" is not a single OS, and you don't really have cross-distro binary compat except by luck, and for the different versions of a same distro they concentrate on rebuilding everything from source, plus you have a unified namespace of symbols in ELFs so you would not be able to load 2 different C++ libs anyway. This brings drawbacks, but also advantages (for ex you can use C++ libraries of the platform, with an unconstrained C++ API)

So the end result is kind of what you say, except if you want things to work really well you should probably even do:

* a specific native package for each distro (maybe × version) * try to dynamically link to the fewest possible libs of the platform, ideally only glibc (and yes, an old enough one), maybe also libstdc++ if you want to support plugins, and maybe others if they come with a compatibility policy (so on a case by case basis) * or target something else than a native distro (ex: flatpak, appimage), although it is sometimes kind of equivalent to yet other distros -- anyway at this point it starts to resemble to how you would distribute a binary for Windows -- so with those possibilities existing it is just that under a GNU/Linux distro you also have the classic/more-integrated ones.

Anyway, Chrome manages to do binary packages for various version of Debian, and probably other distros; likewise for VS Code and Virtual Box. And building for multiple targets is arguably less a pain today than 10 years ago.


>So why can't Visual Studio compile to something that works by default on the Windows of the target audience without requiring them to install those additional DLLs?

Since Windows 10 it ships with Universal C Runtime, which is used by VS2015+. System software uses these runtimes too. For Win7+ it was added as system update, though MS later relented and said it was fine to copy-paste all the api-ms* libraries from the SDK into your application directory so it also works on outdated installations.

In the past it was different though, as system software was compiled against MSVCRT.DLL, but you wasn't supposed to use it and instead use version specific DLLs shipped in "Visual C++ Redistributables". Though going back to Windows 9x there was a time when MSVCRT.DLL was the library that was supposed to be used by applications. See http://bytepointer.com/resources/old_new_thing/20140411_092_...


"Windows 10 it ships with Universal C Runtime", really? Is this a different thing than vc_redist?

Games still require vc_redist.x86.exe or vc_redist.x64.exe to be installed, even on Windows 10. Ancient games also require ancient versions of these to be installed.

I have always found it very strange that these wasnt just included in Windows.


There are two "components" to what used to be `msvcrt.lib`:

- The C runtime library (now `ucrt.lib` AKA the universal C runtime) - The compiler runtime (now `vcruntime.lib`)

The former has a stable API, but the latter doesn't - it changes from one compiler version to the next, as it is used to support various C++ language features.

When these two were combined as `msvcrt.lib`, there was no way for windows to ship it with the OS, because it changed with each VC++ version - they could have shipped historical versions, but it wouldn't really help developers who can't guarantee that the the OS version used by their users is newer than their version of VC++.

Now that these two are split, microsoft can ship the stable universal C runtime with the OS, and programs that only rely on the C runtime do not need a redistributable at all. (This makes things easier for other languages like Rust, or things compiled from GCC, which don't need the VC++ runtime).

Programs that use the VC++ runtime still need to ship the corresponding `vcruntime`, although you still have plenty of freedom in how you do that (eg. bundle DLLs, statically link, etc.) and the compiler runtime itself is smaller than the whole C runtime library.


Great explanation, thanks!

Because of using binary libraries and the ABI can break at any time, in fact ABI stability is something quite recent, introduced in VS 2015 and now plenty of people are asking for them to break it again on VS vNext.

C and C++ runtime libraries aren't an OS API on non-UNIX platforms, each compiler vendor ships their own libraries.

Windows 10 now has C universal runtime library as part of the OS, but that is because nowadays almost everyone only cares about Visual C++ compatibility.

You can have your binaries without extra dependencies, code against Win32 in C without any dependency on ISO C or ISO C++ calls, that is how demoscene competitions work.


> are asking for them to break it again on VS vNext.

Why?


Because some C++20 features can only be fully compliant with an ABI break.

https://github.com/microsoft/STL/issues/1814

https://devblogs.microsoft.com/cppblog/msvc-cpp20-and-the-st...


> If you compile a C++ program with Visual Studio, it gets dynamic dependencies on some visual studio DLLs. DLLs that are not necessarily installed by default on the Windows of the target user you're compiling for.

There is no "necessarily" about it. They are not installed by Windows, period. If they exist on the user's machine, it's because other applications have installed them, for themselves. Applications should have these libraries in their own folders, and those libraries are the specific version the applications were built with.

When you ship an application build with the Microsoft toolchain, you ship the required redistributable libraries in your installer.

Using libraries on Windows is a cinch, because Windows does the smart thing: when an executable needs a DLL, it first looks in the same directory as that executable. Thus your installer can just throw your exe's and dll's into one folder and be done with it.

There is a new thing in Windows: the UCRT (universal C run-time library). It's an attempt to provide something similar to a libc in the Unix-like world: a stable library in the OS, so applications don't have to ship one. Or something of that nature.

Windows has a C library for its own utilities and middleware called MSVCRT.DLL. That is off-limits to applications though. This new UCRT is documented for application use.


Because: https://zachmortensen.files.wordpress.com/2013/08/microsoft-...

> Also, given that Windows comes with pre-installed programs that were written in C++, how did those programs get compiled to not require those Visual Studio dependencies, are MS not using Visual Studio themselves internally?

They do, iirc Windows pretty much always came with "unspecified" versions of the MSVCRT "for internal use only". Also, iirc, at some point Windows took some steps to remove those from the default search paths or something.


Because bundling DLL in your installer is not hard? Because most non-toy program will have DLL of their own anyway that will need installing?

As for the original craziness header, I found it hilariously, but pretty typically, harsh and unfair. Finding the compiler and all is environment is not an easy task in any OS. I've been programming for a looooong time I remember havign to write code that compiled on: IRIX, Solaris, HpUx, IBM unix, Windows and more.

Anyone who ever took a look at configure.sh knows the horrors it holds at bay.


(note this reply might not use accurate terms, but it's just to explain some general principle)

visual studio DLLs.

I assume you mean the C runtime etc, which isn't tied to VS but to 'build tools' i.e. compiler/linker. VS just uses those. It can also use other non-MS compiler/linker.

So why can't Visual Studio compile to something that works by default

It largely can, see next answer.

on the Windows of the target audience without requiring them to install those additional DLLs?

I think (not 100% sure though) because there were historically different versions of those. And as said possibly from different compiler vendors. See also https://stackoverflow.com/questions/16167305/why-does-my-app...

But: things changed somewhat since then though, there's now a 'universal C runtime' which gets installed with Windows 10 and which covers versions of the runtime used per default by the built tools which ship with VS2015 and beyond. Which given the history is quite a piece of work they pulled off.

how did those programs get compiled to not require those Visual Studio dependencies,

Static linking.


> why can't Visual Studio compile to something that works by default on the Windows of the target audience without requiring them to install those additional DLLs?

Not without a time machine; the problem is that you can target older versions of windows with newer versions of Visual Studio. When Microsoft produces vc_redist_2022.exe they can put it Windows 11, but how do they put it in the 2018 version of Windows 10?

The arrival of the Windows Store raises the possibility of automatically installing dependencies. You can also create an "unpackaged" app that uses the WinRT APIs without being a Windows Store app distributed through the store, but if you do that you're exposed to all the horrific machinery yourself as a developer: https://docs.microsoft.com/en-us/windows/apps/windows-app-sd...

In re static linking: I haven't checked but I suspect that COM is involved somewhere. Does anyone have a good summary of what's actually _in_ the redistributable?


COM best practices should be to ship the libraries as registration free components.

https://docs.microsoft.com/en-us/windows/win32/sbscs/creatin...

Unfortunately, like in every best practices from Microsoft, some teams are in deep need of internal advocacy.


> can't Visual Studio statically link in whatever extras it needs

It can but you have to change some options (`/MT` iirc). And if you want to be really minimalist you only need to statically link the runtime (if you avoid using the C++ standard library).


The slight trickiness with this is that if you have any C/C++ libraries that you link into your application, then you need to static link those too. The problem with dynamically linking against a library while statically linking against VS runtime is that you basically end up with two or more copies of the runtime running at once in your application (e.g. two copies of any internal data structures used to manage memory with malloc/free/new/delete).

For similar reasons, you can't have static libraries that are used by higher-level dynamic libraries (e.g. gRPC dynamically linked with protobuf statically linked). In the end, the easiest solution is usually to pick static linking or dynamic linking and use it for everything, although it's possible not to if you're really careful.

Having said that, statically linking everything, including the Visual Studio runtime, works really well and I'd recommend it.


Always link statically whenever possible (looking at you mac os)

We've come a long way since the time Microsoft was almost broken up. I can't believe HN are now calling for Microsoft to be more tightly integrated.

> So the question is: MS created both Windows and Visual Studio. So why can't Visual Studio compile to something that works by default on the Windows of the target audience without requiring them to install those additional DLLs?

Because msvcrt.dll must be compatible with MSVC6 or something ridiculous like that (the amd64 version probably needs to be ABI compatible with a different version, because MSVC6 didn't have amd64). They want to maintain backwards compatibility with existing software, so they can't change the ABI around. You might think, who cares about old software? Well, last I checked MinGW links against msvcrt.dll as well, so to provide compatibility with them (and probably software built today with the MinGW toolchain) they still need to keep it around and compatible.

This is analogous to running with really old glibc… even that had a soname bump (it's not at 1 anymore). Can't exactly add symbol versioning either, since 1) it needs to support older versions of Windows, 2) people could totally roll their own dynamic loader… PE isn't exactly secret.

For your last paragraph, as I understand it the Windows team basically uses their own fork of the toolchain, stuck bring binary compatible…


This lets you build against the MSVCRT that ships with all versions of Windows: https://github.com/neosmart/msvcrt.lib

This has annoyed me going back to Silverlight. They really just don't care users have to install extra things to get your program working.

First-party toolkit stuff like this should be built into the OS.


Sorry, but I disagree with you, this is a packaging issue. The average user is not in a position to make a determination of which toolkit is required for which binary. The only entity with that information is the developer. The developer should make sure all dependencies are either in the OS, or shipped along with the binary.

Java and .NET have entered the chat

I've never quite understood why the .NET runtime didn't ship with Windows. And Microsoft presumably controls Windows Update too, right? :thinking:

A .NET runtime does ship with Windows . It is just often not the version being targeted by a developer which means the targeted runtime needs to be installed.

One reason why some have preferred not to ship with Windows is so that they aren't tied to the Windows release cycle. Not to mention, getting fixes into the monthly updates in a whole thing in and of itself.

The new (.NET Core then .NET 5 onwards) runtimes aren't installed automatically.

But when you install them manually, they are serviced for security updates through Windows Update, so that you don't have to do that manually.


Probably just because they aren't created by the same team. Teams in MS are so horizontally disconnected that you can even find 4+ types of menu in the windows 10 build-in apps. The people created windows and the one handling VS aren't necessary have very strong connection.

You can completely avoid linking it, if you don't use any C or C++ standard library functionality.

MS are not the best at dogfooding. I think they even once released VS as WPF to prove it can be used in a real app.

Anyway, my experience with VC++ and the packages is that you get better control when shipping the dependencies separately. You can resolve the dependencies during installation but it is just more complex.


More than once, that is VS still today since the 2010 rewrite.

Thought it was abandoned but it explains the poor quality of VS and all the constantly resurrecting bugs.

What do you mean?

I feel like VS' GUI is somewhat decent


The GUI in general is ok. Some features are not accessible by shortcuts though which I reported like a hundred years ago.

Along with that, the states of the UI components in VS are sometimes not correct. For instance, the find feature. It changes constantly to Search within open files or. Something nobody ever want to use.

The go forward, go back, feature breaks all the time. I do not even understand how to fail to implement this. Or how to break it. It is basically stack.

...


It's scriptable and you can assign shortcut key combo to anything, so... part ofthe blame falls back to you?

Do you think VS designers should accommodate your particular needs and desire of a particular shortcut over the needs of all other users who participate in their beta and user feedback and instrumentation data?


Not more not less of other IDEs.

True, but then VS is a commercial product and a fairly expensive one. Seems like there are no automatic regression tests at all. Basic functions breaks all the time. Just a very poor quality piece of software for being a software targeting developers.

Ah you mean like the bugs I hit all the time on commercial IDEs like JetBrains products?

This file was about 400 lines before we started adding these comments. You might think that's way too much code to do something as simple as finding a few library and executable paths. I agree. However, Microsoft's own solution to this problem, called "vswhere", is a mere EIGHT THOUSAND LINE PROGRAM, spread across 70 files, that they posted to github unironically. I am not making this up: https://github.com/Microsoft/vswhere

Such a seemingly simple problem and yet we have two very complicated solutions. Just goes to show how convoluted things can get at Microsoft scale.


`vswhere` is for querying a variety of information about all visual studio installs. It does far more than the "simple" problem of finding libraries.

But how do you find vswhere?

I have VS installed on my system, yet vswhere is not in PATH.


Ah, my friend, for that you need `vswherewhere`

> vswhere is included with the installer as of Visual Studio 2017 version 15.2 and later, and can be found at the following location: %ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe

The irony is that there is absolutely no guarantee that this path is stable so in 2025 I'm sure there will be a wherevswhere.exe


There is a guarantee that this path is stable:

https://github.com/Microsoft/vswhere/wiki/Installing

> This is a fixed location that will be maintained.


"Windows 10 will be the last Windows". Guarantees work as long as they don't change their mind.

Find me one place where Microsoft officially documented that there would be no version of Windows past 10.

https://www.pcmag.com/news/windows-10-the-last-version-of-wi...: a developer evangelist said it at their Ignite conference in 2015, and a subsequent official statement by a spokesperson clarified the context of that statement in a way that strongly implied that it was indeed official policy at the time, but with the explicit caveat that it could change again (“We aren’t speaking to future branding at this time”).

I know this quote. It was said once and the official statement was neither a confirmation or a denial. Which makes sense because why would Microsoft want to commit themselves either way?

Eh. They're basically right. Windows "11" is just rebranding. There's no reason why it couldn't have been a Windows 10 feature update, except that they wanted to drop support for older hardware.

`%ProgramFiles(x86)%\Microsoft Visual Studio\Installer`

The is true no matter where Visual Studio is installed to.


Why would they store it under the "Installer" path?

This is a pattern that bothers me with Microsoft software on Windows. They leave the stuff they needed to use ALSO for installer stuff in an installer directory, not under my path, not even in a place I might know of. You could have said

%APPDATA%\VisualStudio\VC2932.2.1.21113412\1231231\a31\Internal\Installer

and it would have been just as easy for me to find.


You can also get it via NuGet, for your build scripts.

Reminds me of something I read earlier mentioning "design by consortium" where the complexity of something goes up because there's so many people working on it and not enough straightforward documentation

And sometimes (which is I think what most applies to Microsoft) too many authorities that must be pleased and nobody with imperial power to say "In the interests of simplifying, we're going to piss off key stakeholders for a period of time." Microsoft's OS team has, at points in the past, rebuilt data structures from C++ to C with very specific memory footprints rather than break, for example, Photoshop (which was getting more performance from running its own OS structure allocator instead of calling the APIs designed to allocate structures).

Simple ecosystems and simple software usually have the same thing in common: they are either young, or they've recently survived an extinction-level-event. Older systems with many dynamic constraints breed complex webs.


> Simple ecosystems and simple software usually have the same thing in common: they are either young, or they've recently survived an extinction-level-event. Older systems with many dynamic constraints breed complex webs.

At the risk of digression, I wish some of the "small government" crowd would realize this.


Yes, the important keyword here is "seemingly". :)

> Just goes to show how convoluted things can get at Microsoft scale.

It's not so much a problem of "scale" as simply poor architecture, neglected technical debt and outright incompetance


I've programmed in some capacity for a long time. I never got into MS programming, except when I was a kid with DOS and some BASICs (gwbasic, mostly).

The reason I've stayed away from MS is that it's always felt inelegant to me. Not only are the systems clunky but the code just looks so ugly to me.

AnExtremelyLongFunctionNameThatWrapsTwiceOnTheScreen(MsI.BillGates.IncrementValueOnlyIfNotInWindows95).

It is not an elegant system by any stretch of the imagination. Same goes for Powershell.


Long function names are often self-documenting in terms of what they're supposed to do, even if not the how. What's more clear?

find_windows_kit_root_with_key

find_kit


The ideal solution is to use modules/classes and parameters names to contextualize, eg:

Windows.find_kit_root(key)

And is even better if your language supports pattern match on the parameters like elixir, so you can "overload" the function with many types of arguments, eg:

Windows.find_kit_root(%User{key: key})

Windows.find_kit_root(%Admin{key: key})

Providing different behaviors to the same function name



Here's the equivalent code in Zig, in case you want a still-maintained, up-to-date file. It's just called "windows_sdk.cpp".

https://github.com/ziglang/zig/blob/master/src/windows_sdk.c...


>"apparently, not all valid Windows file paths can even be converted correctly to UTF-8"

What is the author referring to here?


Windows paths are sequences of utf-16 code units, so they can contain unpaired surrogates which are not legal in unicode (and therefore UTF-8).

That’s why on windows Rust’s OsStr is an extension of UTF8 called WTF8, which allows surrogates (then again “traditional” unix paths are just bags of bytes so it’s not like windows is any worse).


NTFS is old, so I think they're actually just sequences of 16-bit values, once upon a time these were presumed to be UCS-2 characters, now they're presumed to be UTF-16 code units, but either way just as Unix filenames are actually just [u8] the Windows filenames are just [u16]

Bags of bytes is at least the honest, "we know that none of you know jack shit about encodings and if we gave you nice APIs some Go developer will ignore them to avoid dynamic linking so the only safe thing to do is poke file paths as opaque blobs and let people optimistically decode them.

MS's approach is better in the case where we're all good stewards of the filesystem but the edges are sharp when someone isn't.


Windows is actually quite a bit better. The APIs require you to use an encoding. The multibyte `A` APIs can be whatever encoding the current process is using and will be automatically converted to valid UTF-16. The `W` APIs are for UTF-16. The issue is the latter does not enforce valid UTF-16.

So in practice the only way invalid UTF-16 shows up is from malicious programs (or people testing their handling of non-unicode).


The W mode was the only way to do Unicode until relatively recently, A was just legacy pre-Unicode encodings. So there's presumbably a fair chance for older Unicode apps to accidentally mess up the encoding, especially if it's really old stuff from back during the UCS2 era which didn't have paired wide characters (although I suspect at that time most stuff would use A functions in order to work on Windows 9x).

If they were doing conversions manually, I suspect such a major encoding error would have shown up quite quickly. It's bound to cause errors when a file is unopenable in a lot of programs that can't handle non-unicode (heck, Microsoft's own VS code can't handle them).

I doubt even really old NT programs (pre 2000) using UCS-2 would have included unpaired surrogates because IIRC those code points were never mapped to actual characters so I don't think that's an issue in practice.


The problem is not people wilfully introducing lone surrogates but with incorrect string manipulations (e.g. slicing or transformations) not accounting for astrals and thus fucking up on surrogates.

Lone surrogates regularly show up in any environment which allows them.


Windows paths can contain unpaired UTF-16 surrogates, which are not representable in UTF-8.

I wondered exactly the same! Fortunately, the first hit on Google will explain it: https://letmegooglethat.com/?q=not+all+valid+Windows+file+pa...

I solved this problem in 2017 when they changed things around a bit.

CMD /c vcvarsall.bat x64 && set

Then dump the VC specific environment variables in a file and you are good.

I implemented this in tundra a very nice and fast build system. It's in Lua... you can look at it here https://github.com/deplinenoise/tundra/blob/master/scripts/t...


'vcvarsall.bat' is not recognized as an internal or external command, operable program or batch file.

The problem this code is solving is equivalent to finding vcvarsall.bat in the first place.


Which is trivial. For all versions of Visual Studio it's has always been in the same location. They changed the directory structure somewhat around 2017 but it was a minor change.

Also, if you want to run that command you need to run it from the developer command prompt. Otherwise it's location isn't in your path but like I said this script is always in the same location and what it does is that it sets up the environment for you and it's configurable via parameters.


It is decidedly not trivial because people can and do choose to install Visual Studio to any random path they want. You can't assume that it is in C:\Program Files (x86) or %ProgramFiles(x86)% or anything like that, if you care about making reliable software. And there is no guarantee that future versions won't change the directory structure again. In fact there's practically a guarantee that they will someday.

I am well aware that you can use the developer command prompt (in which case you don't need to run that command at all, it would be redundant). But it's a ridiculous and extremely annoying requirement to impose on your users.


wchar_t *value = (wchar_t *)malloc(length);

// The documentation says that if the string for some reason was not stored with zero-termination, we need to manually terminate it. Sigh!

if (value[length]) { value[length+1] = 0; }


You have buffer overflows in your code, `length` and `length+1` are past the buffer end.

That's code copied from the linked gist. But, it omits some key lines in the middle, and does not actually overflow:

    DWORD required_length;
    auto rc = RegQueryValueExW(key, version, NULL, NULL, NULL, &required_length);
    if (rc != 0)  return NULL;

    DWORD length = required_length + 2;  // The +2 is for the maybe optional zero later on. Probably we are over-allocating.
    wchar_t *value = (wchar_t *)malloc(length);
    if (!value) return NULL;

    rc = RegQueryValueExW(key, version, NULL, NULL, (LPBYTE)value, &length);  // We know that version is zero-terminated...
    if (rc != 0)  return NULL;

    // The documentation says that if the string for some reason was not stored
    // with zero-termination, we need to manually terminate it. Sigh!!

    if (value[length]) {
        value[length+1] = 0;
    }
`length` starts at 2 greater than naively needed, then is updated to the real value (again, always 2 less I guess).

That only works if someone does not race and modify the content of that registry key between the two function calls. And that, my friend, is how buffer overflow exploits are born.

Don't poke beyond your end. Don't poke using a value that was returned by a function you don't control. The code shown does both. Such quality, it Jonathan Blow.


Yeah, this function has a few surprising bits of sloppyness ... but:

> NOTE(Kalinovcic): I have translated the original implementation to C

Another point, this is a build-time tool: some assumptions of good-faith input are reasonable and necessary. If an attacker can modify paths to visual studio components in your registry, you have bigger problems (just running the attacker's code directly regardless of safe string handling).


Pray tell, what is the size of a wchar_t on your system?

hah, more than 1 byte for sure, I missed that detail ...

EDIT: I develop on unix systems, not windows, so I'm rusty here ... looking up RegQueryValueExW() the `length` ("lpcbData") is in bytes so everything goes fine until `value[length]` which is indexing by wchar_t (16-bit on windows I guess, 32-bit on linux systems) so it's double the byte offset intended, that's way out.


I’m sure that’s why this person posted this snippet.

wchar_t is also more than one byte, so the buffer is at most half as big as the code appears to expect.


I thought it was parent's code.

Source seems to take into account the buffer overflow:

    DWORD length = required_length + 2;  // The +2 is for the maybe optional zero later on. Probably we are over-allocating.

Does it? If required_length is 10, it allocates 12 bytes, and writes element 13, which is byte 26 and 27.

[in, out, optional] lpcbData (aka &length)

A pointer to a variable that specifies the size of the buffer pointed to by the lpData parameter, in bytes. When the function returns, this variable contains the size of the data copied to lpData.

In other words, it's rewritten to be the actual length before it's incremented for null termination.


`RegQueryValueExW`'s last parameter is in and out, so that `length` is set to the actual written length after the call.

It might cause an OOB write though, with a data race on the registry key (if the key's value happens to grow in length by a char or two between the calls, time of check time of use yada yada).


> It might cause an OOB write though

No, because `RegQueryValueExW` will return ERROR_MORE_DATA and the code bails out on error (also leaking the memory).


1. first call to `RegQueryValueExW` returns a value length of 10

2. length is set to 12

3. external change causes the value to now be 12

4. second call to ``RegQueryValueExW` succeeds, as 12 <= 12, no ERROR_MORE_DATA here; length stays 12

5. length + 1 and length + 2 are now OOB


'by a char or two', they said.

There are two off-by-one errors. Should be checking value[length-1].

There's a comment highlighting this at the bottom of the gist, and apparently an updated version.


> // The fact that we have to do this at all

> // is stupid, and the actual maneuvers we need to go through

> // are just painful. If programming were like this all the time,

> // I would quit.

Yup. Exactly why I refuse to ever work on Windows again, and won't even bother targeting it from other platforms. Total waste of my time and talent.


Well, llvm-config is 719 lines with few comments. At least it doesn't have to use COM and the registry, though.

But llvm-config only works for the exact clang/llvm version it comes with. You can't use it to find any install of clang/llvm on your system.

I am just curious why are there functions/code in a .h file?

This is a single file library. To include it in a project, in the header you put:

  #include "Microsoft_craziness.h"`
Then in a c/cpp file:

  #define MICROSOFT_CRAZINESS_IMPLEMENTATION
  #include "Microsoft_craziness.h"
The define causes the actual functions to be put in, while without it only the declarations are included.


What I don't understand why or where you would use this .h file.

It can't be part of an end-user application, because looking for Visual Studio on an end user's machine is futile.

Maybe if you're writing some kind of extension for Visual Studio?

The comments in the header point to some lack of understanding on the author's part:

  // HOW TO USE THIS CODE
  //
  // The purpose of this file is to find the folders that contain libraries
  // you may need to link against, on Windows, if you are linking with any
  // compiled C or C++ code. This will be necessary for many non-C++ programming
  // language environments that want to provide compatibility.
  //
This needs to clarify the exact situation. Because the normal end-user application cannot be looking for Visual Studio materials. It is completely wrong-headed. They either do not exist, or else were installed by some other application, almost certainly in its own folder. (On this side of year 2000, no application installer should be putting libraries into a system folder.)

The folders that contain libraries you may need to link against are found by default; you don't have to tell your program where to look for user32.dll or kernel32.dll.

Any non-system library is in your own folder: you ship that library with your program, and it goes into C:\Program Files\YourProgram\bin or whatever.

Any .DLL files which are in the same folder as your .EXE will be found by it automatically; there is no need to write code to look for anything.

// We find the place where the Visual Studio libraries live (for example,

That place is nowhere, on 99.999% of the installed Windows machines on the planet.

Unless you mean some other application's C\Program Files\OtherApplication\bin folder, where you have no business looking or using.

  // libvcruntime.lib), where the linker and compiler executables live
  // (for example, link.exe), and where the Windows SDK libraries reside
  // (kernel32.lib, libucrt.lib).
kernel32.lib is a kind of liking stub; the library per se is kernel32.dll, which is a system component.

.lib files are not even required for using .dll files; for instance, you can use .dll files via dynamic FFI in various languages, without a .lib file.

Projects built with Visual Studio generally not have to worry where kernel32.lib has been installed; the tooling takes care of that. Programmers don't have to specify that themselves in project files.

It could be a concern if you use some external build system; it would then need some configuration mechanism to point various toolchain-related variables at all the correct directories. A program using "microsoft_craziness.h" could be built which runs on the that build machine and produces a batch file full of environment variable assignments, or whatever. That I can see.


It was originally written for the Jai compiler.

Right? So it's for tooling.

I can sympathize. Years ago (more than two decades) the situation was already pretty crazy. I managed to use some version of GNU make for building Visual C++ code.

The Microsoft CL.EXE compiler knew absolutely nothing without being told. Unlike /path/to/whatever/gcc, it doesn't know where its headers or libraries are, nothing.

In the Makefile, had to specify all the include and library paths in the SDK installation.

It's like trying to use GCC's cc1 directly or something.

Since that time there have been many more releases of Visual Studio and the SDK, all of them moving targets without a doubt. Yikes!


That’s why you use COM or WinRT with strong contracts.

Doesn't help you when what you need is MSVCRT.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: