
The Linux kernel hidden inside Windows 10 - eDameXxX
https://github.com/ionescu007/lxss
======
vvanders
Fun tidbit is it seems the original reason behind this was Android support in
WinPhone10 but when that was axed they migrated it to Ubuntu on desktop.

If true that's a pretty nifty pivot.

~~~
satysin
Indeed, and the rumours are that Project Astoria worked _too_ well which
scared Microsoft. If Android apps "just worked" on Windows then why would
developers bother using their new Universal Windows Platform.

~~~
Amezarak
Which is really unfortunate.

I can't blame Microsoft for it, but the app-gap between WP/WinMo 10 and
Android is getting untenable for me. I'm thinking about switching back, not
because I think Android is _better_ (I don't think it is at all), but simply
because I'm starting to feel left out when all my family and friends are
using, e.g., SnapChat to keep in touch and I...can't.

It particularly riles me up that there _were_ perfectly functional third-party
Windows Phone Snapchat apps, but Snapchat demanded that they be removed and
still (evidently) declined to make a WP app.

Not to pick on Snapchat, it's not an uncommon story.

If I could even just download the Snapchat APK and sideload it on my Windows
phone, that'd be enough for me. But I can't, since Astoria got killed.

~~~
wangchow
Honestly I use the app-gap as a feature. Not having the fluff is great because
snapchat is making people illiterate. We went from real words to abbreviations
to no words and minimal attention span.

On the other hand now that Uber and bank of america have a universal app it
feels more complete.

~~~
jhoechtl
Why the downvote? This is a legit and conscious comment

~~~
colemickens
Some person doesn't like Snapchat. Their dislike of Snapchat and inability to
install it on their phone due to choice of platform has absolutely nothing to
do with, and absolutely no influence on what phones other people buy or what
apps they install on it.

I have an Android phone and it doesn't have Snapchat on it. I also have
friends with varieties of phones, and Snapchat, and they're very intelligent.
It's donwvoted because it's a pointless "get off my lawn comment" that says
nothing other than "I look down on all Snapchat users".

------
vram22
Microsoft has actually been doing Unix related stuff from way back - a fact
that might not be known to some.

Xenix, then SFU (Services For Unix - for interop - on Win NT or 2K), plus they
had (part of?) a POSIX subsystem around then or earlier- I used it a bit for C
utility dev work on WinNT), etc.

~~~
whyenot
Of course Microsoft then licensed XENIX to the Santa Cruz Operation which
after several years renamed it to SCO Unix. SCO eventually split apart, but
the piece that ended up with SCO Unix renamed themselves the SCO Group. The
SCO Group with (possibly indirect) financial backing from Microsoft, attempted
to use XENIX in a number of copyright infringement lawsuits to attack Linux
vendors.

~~~
kragen
Hmm, there are a variety of inaccuracies in your short statement. Please
carefully read the below so that you can be protected from propagating these
errors in the future.

· SCO Unix, although it incorporated some Xenix code, was very different from
Xenix; it derived a lot from SVR4.

· SCO basically went out of business, because their value proposition was
"Unix, but on a regular PC so you don't have to buy an expensive RISC
workstation." This stopped being a useful value proposition around 1997
because they couldn't keep up with Linux. So they sold the SCO Unix product
line to Caldera, a Linux distributor, in 2001. I'm not 100% sure but I don't
think any employees moved from SCO in Santa Cruz, California, to Caldera,
which was in Utah. Maybe someone stuck around to smooth the transition?

· Microsoft's backing of SCO was direct; they "bought a license" in 2003.
Maybe they also did some indirect backing that I don't know about.

· Caldera basically failed in the Linux distribution space in 2002, and the
investors booted out CEO Ransom Love and replaced him with Darl McBride,
formerly of Novell, IKON, a couple of startups, and Franklin Covey (!).

· The copyright infringement lawsuits weren't based on Xenix. Nobody claimed
Linux had copied from Xenix. Rather, they were based on Bell Labs Unix, from
the 1970s, and AT&T Unix System Labs System V Unix. SCO had supposedly
acquired the copyrights to these sometime in the 1990s, and in fact them
giving permission is how the Lions book was legally republished in 1996 or
1997, but it turned out that in fact what they had acquired was not the
copyright ownership, but a sublicenseable license to the copyright. The actual
copyright rested with some company Novell had bought up at some point, so the
lawsuit got thrown out of court.

· The SCO Group sued not only Linux vendors, notably IBM, but also Linux
users, notably AutoZone and DaimlerChrysler. This is an error of omission, but
it's crucially important.

~~~
whyenot
I appreciate your critique, but stand by my original statement. Let me address
the items you bring up point by point. Please excuse my brevity.

1\. Xenix was based on Bell Labs V7 and then AT&T SVR2, SCO Unix was based on
SVR4. Xenix and SCO Unix shared code in addition to that derived from their
common ancestry. Claiming that they were "very different" is obviously a
matter of opinion. In my opinion, they were pretty similar.

2\. SCO split into two pieces: one was sold to Caldera, the other became
Tarantella. As far as your hypotheses about why SCO failed, I'm not seeing why
it is relevant to the discussion.

3\. Microsoft indirectly backed SCO Group through BayStar capital. Please
verify this for yourself with a quick web search so you will be protected from
propagating your misinformation in the future.

4\. Not relevant.

5\. Both XENIX and SCO Unix were derived from the same original codebase. SCO
Unix would not exist if it were not for XENIX.

6\. Actually, I think the AutoZone and DaimlerChrysler lawsuits are fairly
unimportant within the context of the original discussion. To me, the
interesting point is that MS had the foresight in 1979 to see that Unix would
become a big thing and as a result created Xenix. Decades later, when Unix
(Linux) was indeed a big thing, by proxy, it attempted to use XENIX to fight
back at Unix (Linux) proponents.

~~~
rogerbinns
Regarding #1 that is mashing together two separate products into 1. (I worked
for SCO.)

The history is that Microsoft were producing Xenix - a port from AT&T.
Eventually they decided to stop doing that work themselves as DOS, OS/2, Lan
Manager and similar were of interest. (Note though that they ran their email
infrastructure on Xenix - it was part of their operational business.) Xenix
was handed over to a father and son company in Santa Cruz. They called the
company Santa Cruz Operation so that in phone calls to Microsoft, the
Microsoft folks would think SCO was a branch office not a different company.

Xenix was updated, ported etc, eventually being called SCO OpenServer. That
"SCO Unix" did not have SVR4 in it. Heck it could barely do multi-processor
and similar. In 1995 Novell (owners of the AT&T Unix at that point) then
"handed" SVR4 over to SCO, with the result being called SCO UnixWare. That was
the SVR4 derivative. SCO did this to get into the enterprise space, and
couldn't do the engineering to bring the Xenix derivative there.

Later in the 90s there was a game of musical chairs as Intel announced the
Itanium, and all the Risc chip and Unix vendors formed consortiums. SCO was
part of one with IBM named Project Monterey. It was supposed to have some
Linux compatibility, but as time progressed IBM cared more about Linux (and
AIX) while SCO couldn't keep up with the engineering commitments.

In 2000 all the Unix stuff (OpenServer and UnixWare) went to Caldera. Well it
would have but the deal was so complicated (it was pseudo licensing with money
and royalties going back and forth). Heck Caldera only really wanted
OpenServer. The SEC went "huh" a while later, so a new deal happened in 2001.
There was also a tech downturn, so a simpler deal was done. All that was left
of SCO was Tarantella hence the company rename.

Caldera renamed themselves to The SCO Group a while later. The grounds for
suing IBM were around Project Monterey, although as far as I know neither side
did all they should have. And then since IBM had gone in on Linux, SCO Group
decided to sue claiming IBM had put Unix code into Linux. Novell got involved
because the details of "handing" over mattered.

~~~
kragen
Thank you for clarifying!

Were you around when stuff moved over to Caldera? Were there SCO people who
moved to Utah, or who became Caldera employees in California?

My condolences on having had such a shitty thing happen to such a beautiful
company. I don't ever remember SCO Unix being a particularly great Unix, but
it was pretty solid, and the company was awesome; it's a place I would have
been proud to work if I'd had a chance.

~~~
rogerbinns
I was around during the transition but I'd always been on the Tarantella side
(I was an architect of the product) and so didn't work on the Unix side
directly. Heck our division customers were overwhelmingly running Solaris with
Linux picking up in the 2000's.

The Unix side of the business was rapidly shrinking. 1999 was a banner year
because everyone had to go out and upgrade their operating systems so they
could claim y2k compliance. 2000 was the end of a tech bubble, and also saw a
tech downturn. This lead to drastically reduced sales. Throw in Linux getting
increased adoption (remember that IBM promised to spend $1 billion on it which
gave a lot of credibility), and that SCO's UNIX products no longer had
particularly relevant sweet spots in the market.

I don't know of anyone who moved to Utah. The people in California became part
of Caldera but I don't know exactly how that was legally structured. Also SCO
had folks all over the US and world. At the peak it was ~1,100 employees and
$250m annual revenue.

SCO was a good company and many people liked working at the company because
they liked many of their colleagues. Employee turnover was quite low because
of that. It also had bad points, but what doesn't?

SCO Unix (OpenServer specifically) was great but not from a technological
viewpoint. However the vast majority of users were not techies - they were
dentists, receptionists, pet cemetery workers etc. OpenServer came by default
with a gui that let those regular folk get things done in a friendly way. See
my sibling comment about why Caldera wanted OpenServer.

And SCO did have some firsts. It was also very good at snatching defeat from
the jaws of victory. It was the first company to offer Internet in a Box. You
installed the system, and now were on the Internet (as a server). We were the
first to ship a browser (licensed Mosaic). We shipped by far the most copies
of Netscape. At one point Pizza Hut started allowing orders over the Internet.
It was more of a proof of concept rather than massively used and widespread.
But SCO was behind that too.

------
fapjacks
Honest question: What's the deal with this stuff? I'm not interested in the
reasoning that it's done just because they can do it. I mean... Why not just
... _use Linux_? I suspect it's a comfort zone thing, and most people really
don't want to move out of their comfort zone (and especially programmers have
a hard time admitting this). I understand people developing for the platform,
but I don't understand people developing for Linux on Windows. I hear some
people talk about Visual Studio being a great IDE, but I know _tons_ of people
that don't use VS. They use ST3 on Windows to develop for Linux, and they're
happy about this stuff, and I just don't understand why people wouldn't just
migrate to Linux in the first place.

~~~
willtim
Unfortunately Windows has much better hardware support (e.g. Skylake laptops,
multi monitors via displayport MST).

~~~
fapjacks
Does it, though? I use four monitors with Linux Mint at work while my laptop
is docked. The 'Optimus' dual-GPU thing works out of the box. But I have a
Windows 10 gaming rig at home for Star Citizen, and when I installed Windows,
it didn't recognize the onboard NIC _or_ the AMD 9790 GPU with default
drivers. And as it just so happens, late last night, this same Windows 10 box
mysteriously lost network functionality, and I had to reinstall network
protocol drivers (!). I hear "hardware support" a lot, but I haven't had
problems with hardware support on Linux for a few years at least, and _every_
time I try to use Windows, I have hardware problems. The Windows users on our
dev team seem to have problems regularly. You're right that it _used_ to be a
thing. But I just don't buy it anymore.

~~~
rodgerd
> Does it, though?

Unless you live in a very special bubble, yes.

~~~
witty_username
My desktop, my brother's laptop, and a printer work well with Linux, perhaps
because they aren't high-end.

On the other hand my mother's friend had a mysterious driver problem (screen
glitching) with his NVidia card after upgrading to Windows 10.

~~~
mamon
>> perhaps because they aren't high-end

Or because they are not new. I had a bad experience after i bought, in January
2016, Intel NUC with Skylake processor and Iris 530 graphics card.

I had few months of struggle, like:

\- problem with installation (installer won't boot without some cryptic kernel
parameters passed),

\- lack of graphics driver

\- random crashes (like Google Maps causing the whole system to hang,
requiring hard reset)

\- processor not running at full speed

\- system seeing only one logical core instead of 4 (2 cores x HT)

\- "shutdown" system button causing reboot instead of power off

Most of those were fixed only after Ubuntu 16.04 came out, at the end of
April. Some issues however, persist.

So, my impression is that Linux is good choice only if your hardware is quite
old (like, say, two years, or at least one processor / graphic card generation
behind)

For people like me, who want latest and greatest hardware Linux is not an
option.

------
punnerud
Open PDF on iPhone:
[https://raw.githubusercontent.com/ionescu007/lxss/master/The...](https://raw.githubusercontent.com/ionescu007/lxss/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf)

------
coverband
I'd love to see a video of the demo session for this; really curious how
ProcMon and AV tools will deal with ELF binaries in this release.

~~~
sdrinf
Screenshot: [http://imgur.com/a/5Fcr9](http://imgur.com/a/5Fcr9)

Short summary:

* Ubuntu, and all children apps run under current user's credentials, but are launched under an Lxssmanager servicehost process (as opposed to explorer.exe)

* ProcMon can interact (send sigterm / etc to) unix processes; but as shown on htop, unix processes are not aware of windows ones; nor have I found any ways to send signals backwards

* API compatibility is excellent. Last 2 days, I have re-compiled a full python & elasticsearch & mysql stack into this. Overhead is significantly lower, than any of the virtualization stacks

* I don't use AV tools; however, I suspect they don't check for ELF files. Host system is mounted in /mnt/c , /mnt/d ,etc in LXSS, with current-windows-user creds

* 2-command Sandbox reset: lxrun /uninstall & lxrun /install <\- will restore the linux subsystem to factory default

* LXSS root dir (/usr /var, etc) is in c:\Users\$username$\AppData\Local\lxss\rootfs\ ; home is mounted into c:\Users\$username$\AppData\Local\lxss\home\$username$\

And some caveats:

* Filewatching (specifically, inotify_add_watch ) does not (yet) work

* Manipulating files from the host occasionally makes it _disappear_ (??!) from visibility in the linux subsystem. Specifically, git pull from bash, then git update from host makes git update from bash impossible (index file open failed). Same problems with other types of host file-manipulation. This might be due to permissions, or somethings I haven't figured out yet.

~~~
Zardoz84
And there is some strange behaviour with the keys. Actually using LXSS become
far lees powerfull to the users of Conemu thanks to this. Vim and NeoVim does
weird things with the arrows, on mc I can't navigate on the filesytem with the
cursors, and htop does weird things.

~~~
sdrinf
I suspect that might be conemu, and compatibility thereof. Specifically,
running the bog-standard bash prompt (with conhost.exe) makes this a no-repro;
all keys / arrows works in MC & htop

~~~
GeorgeHahn
Unfortunately, it appears to be a design problem on the Microsoft side:
[https://github.com/Microsoft/BashOnWindows/issues/111#issuec...](https://github.com/Microsoft/BashOnWindows/issues/111#issuecomment-232733125)

------
tracker1
I can't help but think this is all towards an end which is Docker containers
running on Azure's core infrastructure. Having the tooling show up earlier in
Windows 10 as it's being flushed out is just a really nice bonus.

I've been mostly using bash in windows (installed with git tooling) for a
couple years now.

~~~
grennis
That is not Bash on Windows, it's a totally different thing.

~~~
tracker1
I know it's not bash for windows... I was just making a side comment that I
already have been using bash...

------
jimsparkman
So far, this has been fantastic. Our team has been seriously considering
converting from Windows to Linux for embedded development. (we run Linux VM
when needed) Up and running and cross compiling for ARM without issue. Really
surprised how it has just worked so far. This came at a great time for us.

~~~
frik
So far it's been very bad for us. The always-on telemetry in Win10 draws a big
question mark if we can depend on Windows in future. While Windows 7 is great
and around until at least 2020, an expensive move to Linux, FreeBSD or Android
seems like the better option.

~~~
megaman821
If you are trying to escape telemetry Linux or FreeBSD make sense, but not
Android. Between the Android OS and the apps, Android collects much more data
on you than Windows 10.

------
wernercd
Obligatory: Maybe we finally have the year of Linux on the Desktop?

------
sandeep048
Direct link to the slides for the impatient>

[https://rawgit.com/ionescu007/lxss/master/The%20Linux%20kern...](https://rawgit.com/ionescu007/lxss/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf)

------
andai
Link shortcut:
[https://raw.githubusercontent.com/ionescu007/lxss/master/The...](https://raw.githubusercontent.com/ionescu007/lxss/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf)

------
RCortex
Anyone know if it's possible to somehow port the linux subsystem to windows 7
and 8?

~~~
JonathonW
No. The Linux subsystem is dependent on new kernel features introduced in
Windows 10 (pico processes, which were introduced with 8.1 and improved in 10,
improved fork support, file system case sensitivity improvements, memory
management changes, etc.).

~~~
iso-8859-1
How has the fork support improved?

------
ocdtrekkie
I am kinda hesitant to download and open a PDF file coming out of Black Hat
2016.

EDIT: See child comment, GitHub preview is fantastic! Slides have a ton of
great info.

~~~
jacobwg
GitHub will actually render a full preview on the website, no download
required.

[https://github.com/ionescu007/lxss/blob/master/The%20Linux%2...](https://github.com/ionescu007/lxss/blob/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf)

~~~
ocdtrekkie
Ah, that's quite pleasant. And it doesn't even appear to use the browser's PDF
renderer or anything.

~~~
walkingolof
Guys, how do you think Black Hat 2017 is going to be funded if you dont
download and open the PDF ? ;)

~~~
whoopdedo
If they're not able to make it silently download and execute remotely without
me interacting or being aware, I feel they're not doing their jobs as black
hats.

~~~
CamperBob2
The people who know how to do that are pulling down the big bucks on the
Windows 10 team at Microsoft.

------
gdamjan1
is there any more information on this custom init they are having?

------
tobydweymouth
How do we get linex

~~~
fuzzfactor
this looks like it-

[http://www.walkingrandomly.com/?p=6187](http://www.walkingrandomly.com/?p=6187)

------
divyekapoor
Excellent preso. Thank you.

------
eatbitseveryday
I must complain the slides are chalk full of text with very few visuals. It
becomes a paper reading instead of a talk at this point.

------
allendoerfer
I wonder how much further humankind would have progressed by now if "because
it can be done" were not such a big motivation for intelligent people.

Edit: Since everybody took this post seriously, I will, too: We would be
closer to some local maxima, but would have no chance to progress beyond them.

~~~
cmdrfred
The biggest motivation for me is when people tell me that it can't be done.

~~~
fuzzfactor
Not a motivator for me, but not a deterrent. Some of my best accomplishments
were regarded as less than possible by the majority of other operators.

Sometimes the intersection of "because it can be done" & "the most that can be
done with the resources within reach" can be applied in the most problem-
solving of ways.

Haven't tried the Linux in W10 yet, but at least have gotten DOS to boot to
bare metal on a modern PC from a GPT-structured HDD reliably with less monkey
business than ever.

