
Windows file access performance compared to Linux - eatonphil
https://github.com/Microsoft/WSL/issues/873#issuecomment-425272829
======
marsrover
I develop on Windows 10 because I’m a .NET developer and really like the idea
of WSL. But the more I use it the more frustrated I become by its file access
performance. I started off using it for Git but now I just use Git bash in
Poweshell (which also annoys me with its slowness).

I haven’t developed on an actual *NIX machine in years but recently I deployed
something to my DO VPS and it “built” (php composer) in what felt like 1/100
of the time it was taking on my computer, whether running the Windows binaries
in Powershell/CMD or the Linux binaries in WSL. Although I will say WSL is
slower.

In fact, it was so fast that I’m about to call Microcenter and see about
picking up a Thinkpad to install Linux on.

~~~
Waterluvian
Curious just how feasible doing . NET dev in Linux would be? Is it a non
start? Is it inconvenient and rough? Is it fine if you give it enough love?

~~~
akiselev
It's the lack of visual studio that is often the nonstarter.

~~~
platz
You can get by with vscode for a lot, but not everything I think.

~~~
Waterluvian
Is this purely about convenience tools in an IDE or are there some things
actually locked into the Windows environment? Is .NET a lot like Xcode where
it's not like you can just download the libraries + compiler + a text editor
and have all you need? The latter is broadly true for every language I've
really dove into so this feels like a foreign concept.

Edit: what I could stand to gain is that I work weekly in four or five
languages. Already well tooled up in Ubuntu and vscode. Nothing frustrates me
more than having to keep multiple IDEs consistent. Imagine driving two cars
all day that have their controls in all different places.

~~~
paavohtl
Depends on what you're doing. Some technologies are only supported on Windows
(like the official UI frameworks), but most things like webdev and gamedev
libraries are supported on all platforms. Giving up Visual Studio can be a
hard sell, as from my experience C# + Visual Studio (+ ReSharper) is one of
the most productive programming environments you can have.

You can download the SDK from
[https://dotnet.microsoft.com/](https://dotnet.microsoft.com/) and use it on
any platform with your editor of choice, including JetBrains Rider, which is a
cross-platform .NET IDE.

~~~
tracker1
Depending on your UI needs, may want to look at Eto.Forms and MonoGame. :-)
That said, I agree on productivity for C# + VS. I find I'm more productive
with node + npm + vs code though.

.Net Core hasn't been too bad outside VS... I do wish they'd stuck to the JSON
project format. I also wish dotnet had a task runner like npm in it.

------
nickjj
I remember this issue (I commented in it a few years ago).

On the bright side, WSL as a development environment is no longer slow when it
comes to real world usage. I've been using it for full time web app
development for the last year and even made a video about my entire set up and
configuration a week ago[0].

For example with WSL you can easily get <= 100ms code reloads even through a
Docker for Windows volume on really big applications with thousands of files,
such as a Rails app.

Even compiling ~200kb of SCSS through a few Webpack loaders (inside of Docker
with a volume) takes ~600ms with Webpack's watcher. It's glorious.

I haven't had a single performance issue with a bunch of different Rails,
Flask, Node, Phoenix and Jekyll apps (with and without Webpack). This is with
all source code being mounted in from a spinning disk drive too, so it can get
a lot faster.

So while maybe in a micro-benchmark, the WSL file system might be an order of
magnitude slower (or worse) than native Linux, it doesn't really seem to
matter much when you use it in practice.

[0]:
[https://www.youtube.com/watch?v=5gu8wWX3Ob4](https://www.youtube.com/watch?v=5gu8wWX3Ob4)

~~~
quietbritishjim
> with WSL you can easily get <= 100ms code reloads even through a Docker for
> Windows volume

(Edited after watching your video.)

In your video it looks like you're running things _in Docker containers_. Even
if you start containers using WSL, they still run in a separate Hyper-V
virtual machine with a true Linux kernel, whereas WSL shares the Windows
kernel and works by mapping Linux system calls directly to Windows kernel
calls. When you run the "docker" command in WSL, it's just communicating with
the Docker daemon running outside of WSL.

Docker runs this way on Windows because WSL does not implement _all_ the Linux
kernel system calls, only the most important ones needed by most applications,
and the missing ones include some needed to run the Docker daemon.

All in all, this means that what you're talking about is not affected by the
linked issue because it uses a different mechanism to access files (the
Hyper-V driver rather than the WSL system call mapping). Although, if
anything, I would expect Hyper-V to be even slower.

~~~
nickjj
_(Your edit makes my reply make a lot less since since you removed all of your
original questions, but I 'll leave my original reply, read the part after
number 7)_

My set up is basically this:

1\. I use WSL as my day to day programming environment with the Ubuntu WSL
terminal + tmux[0]. It's where I run a bunch of Linux tools and interact with
my source code.

2\. I have Docker for Windows installed (since the Docker daemon doesn't run
directly in WSL yet due to missing Linux kernel features like iptables, etc.).

3\. I installed Docker and Docker Compose inside of WSL but the daemon doesn't
run in WSL. I just use the Docker CLI to communicate with Docker for Windows
using DOCKER_HOST, so docker and docker-compose commands seamlessly work
inside of WSL from my point of view[1].

4\. All of my source code lives on a spinning disk drive outside of WSL which
I edit with VSCode which is installed on Windows.

5\. That source code drive is mounted into WSL using /etc/wsl.conf at / (but
fstab works just as well)[2].

6\. That source code drive is also shared with Docker for Windows and
available to be used as a volume in any container.

7\. All of my Dockerized web apps are running in Linux containers, but using
this set up should be no problem if you use Windows containers I guess? I
never used Windows containers, but that seems out of scope for WSL / Docker
CLI. That comes down to Docker for Windows.

But, it's worth mentioning I have installed Ruby directly in WSL and didn't
use Docker, and things are still just as fast as with Docker volumes. In fact,
I run Jekyll directly in WSL without Docker because I really like live reload
and I couldn't get that to work through Docker. My blog has like 200+ posts
and 50+ drafts, and if I write a new blog post, Jekyll reloads the changes in
about 3 seconds, and that's with Jekyll-Assets too. I have a feeling it
wouldn't be that much faster even on native Linux since Jekyll is kind of
slow, but I'm ok with a 3 second turn around considering it does so much good
stuff for me.

[0]: [https://nickjanetakis.com/blog/conemu-vs-hyper-vs-
terminus-v...](https://nickjanetakis.com/blog/conemu-vs-hyper-vs-terminus-vs-
mobaxterm-terminator-vs-ubuntu-wsl)

[1]: [https://nickjanetakis.com/blog/setting-up-docker-for-
windows...](https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-
wsl-to-work-flawlessly)

[2]:
[https://github.com/nickjj/dotfiles/blob/master/etc/wsl.conf](https://github.com/nickjj/dotfiles/blob/master/etc/wsl.conf)

~~~
tracker1
It's worth noting that copying in/out of the docker for windows container can
be interesting. Docker for windows mirrors the content in the volume directory
on windows to one native inside the container... for some editing, it's
fine... but try something a SQLite database with a gui in windows, and a
container in linux both connected, and it will blow up on you.

The mount in WSL is really adaptive calls to the native OS, so that side will
work without issue... the sync inside the container while editing is fast
enough, and the running/building in the container is faster than it would be
on the WSL itself as described by GP.

------
herf
I spent many years optimizing "stat-like" APIs for Picasa - Windows just feels
very different than Linux once you're benchmarking.

It turns out Windows/SMB is very good at "give me all metadata over the wire
for a directory" and not so fast at single file stat performance. On a high-
latency network (e.g. Wi-Fi) the Windows approach is faster, but on a local
disk (e.g., compiling code), Linux stat is faster.

~~~
avar
I'm mostly clueless about Windows, so bear with me, but that makes no sense to
me.

If SMB has some "give me stat info for all stuff in a directory" API call then
that's obviously faster over the network since it eliminates N roundtrips, but
I'd still expect a Linux SMB host to beat a Windows SMB host at that since FS
operations are faster, the Linux host would also understand that protocol.

Unless what you mean is that Windows has some kernel-level "stat N" interface,
so it beats Linux by avoiding the syscall overhead, or having a FS that's more
optimized for that use-case. But then that would also be faster when using a
SMB mount on localhost, and whether it's over a high-latency network wouldn't
matter (actually that would amortize some of the benefits).

~~~
a1369209993
I think the idea is that you're accessing files sparsely and/or randomly.

With the linux approach you avoid translating (from disk representation to
syscall representation) metadata you don't need, and the in-memory disk cache
saves having to re-read it (and some filesystems require a seek for each
directory entry to read the inode data structure, which can also be avoided it
you don't care about that particular stat).

With the windows approach, the kernel knows you want multiple files from the
same directory, so it can send a (slightly more expensive) bulk stat request,
using only one round trip[0]. On linux, the kernel doesn't know whether you're
grabbing a.txt,b.txt,... (single directory-wide stat) or foo/.get,bar/.git,...
(multiple single stats that could be pipelined) or just a single file, so it
makes sense to use the cheapest request initially. If it then sees another
stat in the same directory, it might make a bulk request, but that still
incurred a extra round trip, and may have added useless processing overhead if
you only needed two files.

TLDR: Access to distant memory is faster if assuptions can be made about your
access patterns, access to local memory is faster if you access less of the
local memory.

0: I'm aware of protocol-induced round trips, but I don't think it effects the
reasoning.

~~~
smartstakestime
just think of they way the OSs are used.

get to a directory:

linux: cd /dir (no info) windows: open directory ... all the info and
different views depending on your selection currently like image file
thumbnails

in windows you are always accessing this meta data so it makes sense to speed
it up. while in linux even the ls fucking does give you meta data you have to
add the extra options so it doesnt makes sense to speed up and waste storage
on something that is infrequent

seem like both ways are sound

------
Solar19
This is interesting, and to some extent, expected. I'd expect that emulating
one OS on top of another is going to have performance challenges and
constraints in general.

But the deep dive into aspects of Windows and Linux I/O reminded me that I'd
love to see a new generation of clean sheet operating systems. Not another
Unix-like OS. Not Windows. Something actually new and awesome. It's been a
long time since Unix/BSD, Linux, and Windows NT/XP were introduced.

A few years ago, Microsoft developed a new operating system called Midori, but
sadly they killed it. MS, Apple, and Google are each sitting on Carl Sagan
money – billions and billions in cash. Would it hurt MS to spend a billion to
build an awesome, clean-sheet OS? They could still support and update Windows
10 for as many years as they deemed optimal for their business, while also
offering the new OS.

Would it cost more than a billion dollars to assemble an elite team of a few
hundred people to build a new OS? Would Apple, Microsoft, or Google notice the
billion dollar cost? Or even two billion?

If you think there's no point in a new OS, oh I think there's a lot of room
for improvement right now. For one thing, we really ought to have Instant
Computing in 2018. Just about everything we do on a computer should be
instant, like 200 ms, maybe 100 ms. Opening an application and having it be
fully ready for further action should be instant. Everything we do in an
application should be instant, except for things like video transcoding or
compressing a 1 GiB folder. OSes today are way, way too slow, which comes back
to the file access performance issue on Windows. Even a Mac using a PCIe SSD
fails to be instant for all sorts of tasks. They're all too damn slow.

We also need a fundamentally new security model. There's no way data should be
leaving a user's computer as freely and opaquely as is the case now with all
consumer OSes. Users should have much more control and insight into data
leaving their machine. A good OS should also differentiate actual human users
from software – the user model on nix is inadequate. Computers should be able
to know when an action or authorization was physically committed by a human.
That's just basic, and we don't have it. And I'm just scratching the surface
of how much better security could be on general purpose OSes. There's so much
more we could do.

~~~
013a
Google is doing this, very publicly, with Fuchsia. Brand new kernel, not even
POSIX compliant.

Microsoft is also doing this, in a different and substantially more expensive
way [1]. Over the past several years they've been rewriting and unifying their
disparate operating systems (Windows, Phone (before the fall), Xbox, etc) into
a single modular kernel they're calling OneCore. Its more than likely that
this work is based off of, if not totally contains, much of the NT kernel, but
its the same line of thinking.

There is one massive rule when it comes to engineering management we see
repeated over and over, yet no one listens: Do Not Rewrite. Period.

Apple is exemplary in this. We don't know how many changes they've made to iOS
since its fork from MacOS long ago, which was based on BSD even longer ago.
But have you used an iPad in recent history? Instant app starts. No lag. No-
stutter rendering at 120fps. When HFS started giving them issues, they swapped
it out with APFS. Apps are sandboxed completely from one-another, and have no
way to break their sandbox even if the user wants them to. Etc. Comparing an
iPad Pro's performance to most brand new Windows laptops is like fighting a
low-orbit laser cannon with a civil war era musket. They've succeeded, however
they managed to do that.

Point being, you don't rewrite. You learn, you adapt, and you iterate. We'll
get there.

(And if you've read all this and then wondered "but isn't Fucshia a rewrite"
you'd be right, and we should all have serious concerns about that OS ever
seeing the light of day on a real product, and its quality once that happens.
It won't be good. They can't even release a _passable_ ChromeOS device [2])

[1] [https://www.windowscentral.com/understanding-windows-core-
os...](https://www.windowscentral.com/understanding-windows-core-os-and-
polaris)

[2]
[https://www.youtube.com/watch?v=HOh6d_r63Bw](https://www.youtube.com/watch?v=HOh6d_r63Bw)

~~~
contextfree
onecore isn't really about the kernel, but the intermediate layers above it.

------
giancarlostoro
> The NT file system API is designed around handles, not paths. Almost any
> operation requires opening the file first, which can be expensive. Even
> things that on the Win32 level seem to be a single call (e.g. DeleteFile)
> actually open and close the file under the hood. One of our biggest
> performance optimizations for DrvFs which we did several releases ago was
> the introduction of a new API that allows us to query file information
> without having to open it first.

Ouch that sounds painful... Is this why deleting gigs worth of files takes a
bit? I could of sworn it's not a huge difference on Linux, at least when using
the GUI, maybe when doing straight rm it's quicker.

~~~
magicalhippo
Reminds me of a performance optimization I did at work. A previous developer
had implemented a routine to clean up old files. The routine would enumerate
all the files in a directory, and then ask for the file age, and if old enough
delete the file.

The problem was that asking the file age given a path caused an open/close, as
GetFileSizeEx expects a handle.

Now, at least on Windows, enumerating a directory gets[1] you not just the
filename, but a record containing filename, attributes, size, creation and
access timestamps. So all I had to do was simply merge the directory
enumeration and age checking.

The result was several orders of magnitudes faster, especially if the
directory was on a network share.

[1]: [https://docs.microsoft.com/en-
us/windows/desktop/api/minwinb...](https://docs.microsoft.com/en-
us/windows/desktop/api/minwinbase/ns-minwinbase-_win32_find_dataw)

~~~
tom_
POSIX stuff is terrible for this sort of thing, but luckily it's often a
straightforward fix once you know to look for it. Here's a similar fix for the
old Win32 version of the silver searcher:
[https://github.com/kjk/the_silver_searcher/pull/7](https://github.com/kjk/the_silver_searcher/pull/7)

Looks like the active Windows port
([https://github.com/k-takata/the_silver_searcher-
win32](https://github.com/k-takata/the_silver_searcher-win32)) doesn't have
this change, but it's possible Mingw32 or whatever exposes this flag itself
now (as it probably should have done originally).

------
jclay
Thanks for sharing, that was an interesting read. I know the IO performance
has always been the main gripe with WSL.

It makes me think more broadly about some of the trade offs with the
discussions that happen on Github Issues.

It’s great that insights like this get pulled out in the discussions, and this
could serve as excellent documentation. However, the discoverability will be
difficult and it always takes some weeding through the low quality comments to
piece the insights from contributors together.

I wonder how Github (and Microsoft) could better handle this. They could allow
for flagging in-depth comments like this into a pinned section and they could
be updated collaboratively in the future (Wiki?).

It also feels like a reputation system could help to motivate healthier
discussions and could bury low quality “me too” comments and gripes. It’s not
so bad in the linked example but I often encounter many rude comments aimed
against volunteer OSS developers.

~~~
Analemma_
This particular post is pretty famous if you've been following the development
of WSL. It's constantly linked to and referenced from other GitHub issues
regarding WSL performance. So I think GitHub's current system is succeeding in
that regard, although there are so many good points raised here that I wish it
could get turned into a standalone blog post.

------
setquk
I can’t see MFT contention mentioned once. That’s what absolutely and totally
destroys small file write performance. This affects source operations, WSL,
compilers, file storage, everything.

And that’s so architecturally tied to the guts of NT you can’t fix it without
pushing a new NTFS revision out. Which is risky and expensive.

Which is incidentally why no one at Microsoft even seems to mention it I
suspect and just chips at trivial issues around the edges.

Bad show. NTFS is fundamentally broken. Go fix it.

Edit: my experience comes from nearly two decades of trying to squeeze the
last bit of juice out of windows unsuccessfully. My conclusion is don’t
bother. Ext4, ZFS and APFS are at least an order of magnitude more productive
and this is a measurable gain.

~~~
Zardoz84
Perhaps we didn't read the same article. What it says that the root of problem
is the Windows IO subsystem architecture. Change NTFS for anything and you
will get the same problem.

~~~
setquk
But that’s not the case. The root cause is the MFT and NTFS architecture.
People fail to mention that because the problem is harder to fix. It’s almost
that there is a “do not speak bad of NTFS” going on.

You can demonstrate this by using a third party file system driver for NT.
when NTFS on its own is eliminated the performance is much much better. This
is a neat little differential analysis which is conclusive. I can’t remember
the product I used when I evaluated this about 8 years ago unfortunately.

------
quelsolaar
I think this is a very good example of how windows is different in its goals
and designs from Linux. I have a feeling this is the reason Linux has had a
hard time catching on, on the desktop. Its easy to complain about a slow
filesystem, but Microsoft lives in a different world, where other priorities
exist. For someone building a server running a database serving loads of
people Linux is a no brainer. You can pick the parts you want, shrink wrap it
and fire it up without any fat. On a desktop machine, you want to be able to
update drivers in the background without rebooting, you want virus scanners,
and you want to have driver ready the moment the user plug's in a new device.
Both Windows and Linux is for the most part very well engineered, but with
very different priorities.

~~~
laumars
I’m very confused by your post. You start off talking about desktop machines
but NT was actually engineered for servers and then later ported to the
desktop. You then describe a bunch of features Linux does better than Windows
(eg updating drivers without a reboot).

I think a more reasonable argument is to make is just that Windows is
engineered differently to Linux. There’s definitely advantages and
disadvantages to each approach but ultimately it’s a question of personal
preference.

~~~
acdha
NT was architected to handle high-end workstations from day one — there’s a
reason why running the GUI was mandatory even when the resource costs were
fairly substantial.

Check out e.g.
[https://en.wikipedia.org/wiki/Windows_NT_3.1](https://en.wikipedia.org/wiki/Windows_NT_3.1)
for the history of that era. The big selling point was that your business
could code against one API everywhere, rather than having DOS PCs and
expensive Unix, VAX, etc. hardware which was completely different and only a
few people on staff were comfortable with.

~~~
laumars
OS/2 was a high end desktop OS, but NT diverged a little and took some heavy
design principles from VMS (hence it’s name, WNT) and was thusly pivoted
towards back office systems rather than desktop usage.

At that time Microsoft’s server offering was a UNIX platform, Xenix, but it
was becoming clear that there needed to be a platform to serve workstations
that wasn’t a full blown mainframe. So Microsoft handed Xenix to SCO to focus
on their collaboration with IBM so the intent there was always to build
something more than just high end workstation. And Given it was intended to be
administrated by people who were Windows users rather than UNIX grey beards
(like myself) it clearly made sense to make the GUI a first class citizen; but
that doesn’t mean it was sold to be a desktop OS.

~~~
acdha
My point was that it is misleading to say it was billed as a server OS when
all of their messaging was that it was both — maybe not as far down as low-end
desktops but they were very clear that any serious work could be done on NT,
going after the higher end PC and lower end workstation business.

~~~
laumars
In that era workstations weren’t the same things as desktops. They were an
entirely different class of computers and often workstations just ran server
OSs with a nicer UI (Next, SGI, etc). So you’re point about workstations
doesn’t invalidate what I was saying about NT not originally targeting
desktops.

------
jclay
They mention that file performance decreases with the numbers of filters that
are attached to the NTFS drive. Is there a way to list the filters to
determine which ones (and how many) are applied on your system?

~~~
AbacusAvenger
I had the same question. Apparently there are a couple of commands to look at:

    
    
      fltmc filters
    

That shows the filters loaded, and presumably the ones with "num instances" of
0 aren't actually in use on any volumes.

You can figure out what each of the filters is by looking at the output of:

    
    
      sc query type=driver
    

To figure out what the ones I saw in fltmc were, I used this:

    
    
      sc query type=driver | grep -A1 -i -e cldflt -e wcifs -e storqos -e filecrypt -e luafv -e npsvctrig -e wof -e fileinfo

------
tahoemph999
Interesting this article never said "poorly architected". The conclusion that
the issue is peanut buttered points at that. Instead of looking into the
system for things to optimize is there any proposal or initiative to rework it
at a higher level?

~~~
tedunangst
Proposal from an earlier comment:

> Noting, as an aside, that it isn't really all that necessary for MSFT to
> work on the project, because I gather there are at least 684 FOSS NT kernel
> developers available, and qualified, and willing, to volunteer their time to
> work on projects like that. I assume that's why all those people upvoted,
> anyway. With a team that size, stabilizing WinBtrfs will take no time at
> all.

------
Abishek_Muthian
On similar lines, one of the usual gripe with windows is the horrible file
copy estimates, Raymond Chen wrote an article on it to explain why it is[1].

[1]:
[https://blogs.msdn.microsoft.com/oldnewthing/20040106-00/?p=...](https://blogs.msdn.microsoft.com/oldnewthing/20040106-00/?p=41193/)

~~~
blntechie
The Old New Thing is a fantastic blog especially if you develop in Windows. I
used to read it religiously until 5 years back and stopped reading about time
when Google Reader was retired and somehow didn't setup in my new flow of
reading blogs. Thanks for reminding me about this blog and article again.

------
Thaxll
At least windows doesn't freeze your whole desktop under heavy IO.

Edit: I'm getting downvoted by people that never used Linux probably.

~~~
newnewpdro
You're being downvoted, but it's true about linux for at least two reasons to
this day:

1\. dm-crypt threads are unfair to the rest of the system's processes [1]. On
dmcrypt systems, regular user processes can effectively raise their process
scheduling priority in a multithreaded fashion by generating heavy IO on
dmcrypt storage.

2\. Under memory pressure, the VM system in linux will enter a thrashing state
even when there is no swap configured at all. I don't have a reference on
hand, but it's been discussed on lkml multiple times without solution. I
suspect the recent PSI changes are intended as a step towards a solution
though. What happens is clean, file-backed pages for things like shared
libraries and executable programs become a thrashing set resembling anonymous
memory swapping under memory pressure. As various processes get scheduled,
they access pages which were recently discarded from the page cache as
evictable due to their clean file-backed status when other processes ran under
pressure, and now must be read back in from the backing store. This ping-
ponging continues dragging everything down until either an OOM occurs or
pressure is otherwise relieved. This often manifests as a pausing/freezing
desktop with the disk activity light blazing, and it's only made worse by the
aforementioned dmcrypt problem if these files reside on such volumes.

[1]
[https://bugzilla.kernel.org/show_bug.cgi?id=199857](https://bugzilla.kernel.org/show_bug.cgi?id=199857)

~~~
rikkus
This behaviour - and perhaps other causes with similar issues for desktop
users, is what drove me away from helping to make Linux ‘ready for the
desktop’. The kernel philosophy was incompatible with the needs of desktop
users. Those developing for the desktop (like myself) didn’t have the required
expertise to make the kernel do the Right a Thing, and couldn’t find enough
people willing or able to help.

Things may have changed over the years - I’ve been running a Linux desktop
recently and haven’t seen this kind of issue yet (the kind where you need to
use magic keys to ask the kernel to kill all, sync and reboot) but reading
your post, perhaps this is because RAM is much more plentiful these days.

~~~
zanny
Its probably the ram plentiful thing. I haven't looked recently into if the
Arch mainline kernel kconfig is just bad or not but the oomkiller is trash for
me. Used to have the "recommended" swap == ram size but then the memory
manager never even tried to cull pages until it was OOM and froze up trying to
swap constantly. Currently running a 16/4 spread and probably going to drop to
16/1 because any time I hit memory limits everything just freezes permanently
rather than the oomkiller getting invoked. I've hit it twice this week trying
to render in Kdenlive and run a debug build of Krita...

------
dustinmoorenet
The issue seems to be with how Windows handles file system operations. They
allow filters to operate over each request. These filters are like hardware
drivers in that they are created and maintained by third parties. So, MS has
no real ability to change how things work because they don't have absolute
control in the same way that Linux does over the operations (Linux device
drivers are part of the kernel)

------
mehrdadn
Not sure if I'm being naive, but couldn't they make it so that if there are no
filters present on a given volume, then the entire filter stack is bypassed
and the directory nodes are served from a cache managed by the file system at
the root of the volume? That way developers who really want performance can
make sure they have no filter drivers installed on that volume.

~~~
tracker1
If they really wanted to, they could mount an ext4 fs as a single blob on disk
directly into WSL. That won't really help if you want to run a GUI editor in
windows against that identical directory though.

~~~
mehrdadn
Any idea why they haven't responded to people's requests for FUSE? It's the
second-most requested feature and should enable applications like this.

~~~
zmj
They have: [https://docs.microsoft.com/en-
us/windows/desktop/projfs/proj...](https://docs.microsoft.com/en-
us/windows/desktop/projfs/projected-file-system)

~~~
mehrdadn
That looks awesome (thanks for sharing) but it seems to be a different thing?
People are looking for a way to run existing Linux FUSE in SL, not develop new
virtual file system implementations. They haven't responded to that request at
all. [https://wpdev.uservoice.com/forums/266908-command-prompt-
con...](https://wpdev.uservoice.com/forums/266908-command-prompt-console-
windows-subsystem-for-l/suggestions/13522845-add-fuse-filesystem-in-userspace-
support-in-wsl)

~~~
int_19h
Windows expects different things from an FS, so it makes sense for a native
userspace FS API to be different from FUSE.

But once you have it, it shouldn't be difficult to write a FUSE adapter on
top, mapping across those differences as best as possible.

------
CodeArtisan
NTOS (Windows NT's kernel) was originally designed to be backward compatible
with Windows (DOS era), OS/2 and POSIX. Designing such a kernel was considered
a performance at the time but probably cost its fair share of bloat that kept
growing with the years. Also, it's not surprising that Linux is optimized for
operating on files since the UNIX philosophy is about considering everything
as a file, something that Dave Cutler (NTOS main designer) was known for
criticizing[1].

 _I pray daily that more of my fellow-programmers may find the means of
freeing themselves from the curse of compatibility._ (Dijkstra)

[1]
[https://en.wikipedia.org/wiki/Dave_Cutler#Attitude_towards_U...](https://en.wikipedia.org/wiki/Dave_Cutler#Attitude_towards_Unix)

------
yellowapple
As another data point here: I had to add a TKinter-based loading screen to a
PyQt5 app I wrote for work because the app - which takes half a second to
start up on Linux - takes nearly a minute on Windows the first time around
(and multiple dozens of seconds on subsequent runs) and I know firsthand that
my end-users are unlikely to wait that long unless they can see something
loading. I suspect it has to do with the sheer number of file accesses for
each of the Python libraries and underlying DLLs and such.

------
qwerty456127
> A Win32 path like C:\dir\file gets translated to an NT path like
> \??\C:\dir\file, where \??\C: is a symlink in Object Manager to a device
> object like \Device\HarddiskVolume4

Cool! Can the "NT paths" be used directly on Windows? As far as I know the
ordinary Win32/NTFS system doesn't even have symlinks and this feels quite a
handicap.

~~~
beart
Windows does have symlinks and file junctions. Not sure how these compare to
other file systems.

~~~
watersb
These require user-mode round trips for path traversal, so comparable to FUSE,
I believe.

So it is relatively easy to add hooks of your own that get called as if they
were file reads; NTFS calls these "Reparse Points". Linux would do this with
named pipes.

I am just a user of these things, haven't dug into any implementation details
recently, but I guess that the stacks to support these features in Linux and
Windows each have their cache etc design decisions that have led to current
state of filesystem performance under typical use-cases these days.

------
mey
I still scracth my head as to why doing a kernel update of Ubuntu running
under Hyper-V on a spinning disk is so horrifically slow. If I migrate the VM
while it's running to SSD, let it finish, then migrate it back to spinning,
it's faster then just letting it update in place. This is my device desktop,
otherwise I would put in on SSD full time.

The above comments make me think something like the stat calls may be the
issue and moving the running vm to SSD hides the problem. It obviously isn't
raw disk rate at the point.

~~~
AaronFriel
Is the Ubuntu VM's / on a fixed or dynamic VHD? The former provides (modulo
filesystem fragmentation) linear addressing of virtual LBAs to physical LBAs,
and the latter can be heavily fragmented and result in sequential read/write
operations becoming random access, which on a HDD kills performance.

My advice for anyone running VMs is if they're on HDDs or they're highly tuned
for disk usage (databases for example), use fixed VHDs, otherwise use dynamic.

~~~
mey
Thanks for the suggestion. The disk is a dynamic VHDX, migrated it to fixed
vhdx and will experiment.

------
Daniel_sk
The build speed difference for Android (Gradle build system) between Windows
and Linux is extremely noticeable especially for large projects, where you can
see even almost 50% faster builds.

------
kuroguro
Wouldn't holding project files in an ext partition fix this? Or is that not
supported yet?

~~~
anticensor
Use ReactOS driver if that works. If not, file a bug report on
[https://jira.reactos.org](https://jira.reactos.org).

------
edoo
The entire post negates the first statement that it isn't easy to just say
"NTFS is slow".

------
bytematic
I would love a QT-esque rust library that utilizes components and maybe even
scss or css for styling

------
tomohawk
tldr: discussion is about why disk access under WSL is slower than under
Linux, mostly due to the different design constraints of Linux vs Windows for
file systems.

------
ezoe
An interesting comment from insider. We all knows from various benchmarks that
Microsoft Windows' filesystem access performance was way too worse than Linux
kernel not only WSL but also Win32 Subsystem too.

Also, process creating performance is worse on Windows. I wonder if it is also
the case of "Death by thousand cuts"

------
ulkesh
Windows performance is abysmal. Instead of beating around the bush, they
should just state that it is the combination of everything that makes Windows
dog slow compared to either Mac or Linux. Someone said it on that thread, but
Windows really should be a DE on Linux now instead of being its own OS.

------
stevefan1999
Perhaps this explains why Yarn/NPM was slow on WSL too?

------
reubenmorais
dyt

