Hacker News new | past | comments | ask | show | jobs | submit login
A Kernel Engineer at Microsoft's Answer to “What Do You Think about ReactOS?” (quora.com)
532 points by whereistimbo 16 days ago | hide | past | web | favorite | 275 comments



Thanks for the link, o for a bunch of new followers on Quora today and was wondering why :)

There are lots of good answers in this thread. In particular, someone took the time to look at the code, and that person wrote it only took them minutes to find similarities that make my opinion difficult to dismiss.

About the lack of documentation, consider that only a tiny fraction of the kernel’s internals surface in the DDK. All our internal headers have a .w extension and are littered with directives like begin_ntddk, end_ntddk (and many others), so a large number of structures and many fields inside existing structures are hidden or replaced by a generic reserved field.

Macros names, parameters, etc. never appears in the compiled code. It is highly improbable (I studied math, “impossible” is not a word lol), almost surely impossible that a clean-room reimplementation ends up using macros for the same things, let alone macros with the same or similar names.

Also, consider that optimizing compilers emit code that is extremely hard to follow even for us. Heck, the debugger, together with the PRIVATE symbols that has the name of all private symbols, and the actual source code, has a hard time to sync with the disassembly with the code when stepping through. While it is possible to write code that behaves similarly seen from outside, it is again implausible that the expression of this behavior results in code that looks nearly identical to the original. Consider that the name of local variables is never part of the binaries, only public symbols are. Also, consider that the compiler aggressively optimizes out variables to reduce memory accesses and holds values in CPU registers as much as possible, so those variables, while conceptually present, don’t really exist as such in the disassembled code. How to explain that a reimplementation comes up with the same variables, declared in the same order, when those variables are optimized out by the compiler?

What about in-line functions, both explicit and auto-unlined by the compiler? How do you even know there was a function in the first place, and how do you invent a name that is identical to the name of said function in the original code?

Funnily, I had a conversation with a very seasoned kernel engineer (I report directly to him) about ReactOS and my Quora reply. He told me the team looked into ReactOS some time ago and reached the exact same conclusions: impossible.

In particular, this person distinctly remembers a hack he implemented (I’m not going to reveal any details, but suffice to say it was in response to some assertion by some 3rd parties that something Microsoft declared in a court of law as very difficult). He explained the hack to me in full details and, boy, hacky that was, and they found the same hack in ReactOS’s code, except that the présumer authors of that “clean room” implementation probably have no idea regarding why the hack was there.

Finally, “clean room” takes another sense when one knows that Alex (yes) worked for Microsoft until spring 2019 at least, as a contractor for a company called Cloudbase Solutions SRL. His Microsoft email address was v-alione@microsoft.com. I don’t know if he had access to the ntos code, or NTFS or anything else, but very close from home he was, for sure.

So to those who want to take my opinion to court, I say it’s a “careful what you wish for” type of thing, but again I’m not a lawyer, and in other regards ReactOS aligns with a very old version of the NT kernel. It is possible (my opinion only) that Microsoft does does not care?


> Consider that the name of local variables is never part of the binaries, only public symbols are.

"Never" and "only public" are wrong in the statement above, because non public symbols were indeed released by Microsoft.

I guess you are young enough not to know that Microsoft accidentally did release some NT builds with the names of the internal variables, and such builds were intentionally made with less compiler optimizations, allowing for easier reversing. Such events of releasing the internal names resulted in some very interesting stories and statements:

https://en.wikipedia.org/wiki/NSAKEY

"_NSAKEY was a variable name discovered in Windows NT 4 Service Pack 5 (which had been released unstripped of its symbolic debugging data) in August 1999 by Andrew D. Fernandes of Cryptonym Corporation."


Also, the Windows code is under shared source license for nearly 20 years. Not really some sensitive secret thing or crown jewels that many want us to believe.


Sure and the source code has also been leaked before. But a "clean room" implementation is not allowed to look at such things.


Back in NT3.x times, for quad-processors machines, and IIRC even NT 4.x with 8-processors machines (like AXEL crazy SMP monsters) Microsoft shipped windows NT as code, and you had to compile it on the machine it was to run on.


Microsoft does not release the kernel’s private symbols, trust me on that. But yes there was some leaks in the past, small portions of NT4 and W2K were leaked, I think I link to a Google query pointing to articles discussing the leaks in the Quora reply.


Well that's categorically untrue. Sure they don't release private symbols intentionally but they have done in the past accidentally. At that point it becomes a bit of a grey area, undoubtedly leaked/stolen source code is a no-no but reversing from private symbols when they do leak seem harder to quantify as you still need to reverse engineer the code, just structures/names etc are already known.

Private symbols are not the only way of gleaning more information, other examples I can think of are:

* Checked builds (prior to Win10). These builds shipped de-optimized kernels (e.g. no inlines) typically with copious debug strings which gave away important details. For example I gleaned a lot of knowledge of ALPC MSRPC from the checked build of rpcrt4.dll from Windows 8.

* SDK/DDK headers, especially in the brave new world of insider previews with preview SDK/DDKs there is sometimes information present which should not have been released including "private" information. Again bit of a grey area.

* The private symbols MS do ship. For example a significant proportion of the COM runtime has private symbols, intentionally. You can extract from those a surprising amount of system call structure information.

I'd recommend watching Alex Ionescu's talk at OffensiveCon about how he does reverse engineering on Windows to see many of these things in action. https://www.youtube.com/watch?v=2D9ExVc0G10

I'm not saying any of this would make it a clean-room re-implementation but to say ReactOS cannot possibly have been reverse engineered without just up and copying source isn't true.

edit: Formatting.


I’d love that you point to an instance where the private symbols of the kernel actually shipped?

It is very possible that some private symbols were part of some leak, but stolen data does not qualify as “shipping” :)

Again, I stand behind my opinion. I eyeballed some of the code side-by-side and there was portions where I could literally see a line-by-line correlation, which I can hardly explain.

Then if reversing the kernel is so doable using legitimate means, why ReactOS is still largely stuck in the early 2000’s, coincidentally where the major leaks happened?


The private symbols have in the past ended up on the public symbol server (and quickly taken down), they have ending up "shipping" in public symbol packs. I can't point to specific incidents as the links to them no longer exist. This is why I said accidentally as they were not released as a conscious effort on MS's part.

However you seem to want to claim the only place those symbols can come from is being stolen. Of course in this case you use leak as a synonym for stolen, bit leak can just as much mean they were released accidentally by the owner, MS can't steal their own private symbols and release them on the web. I'm sure there's some symbol files traded in private scenarios which are actually taken through non public means but there have been actual incidences of public release of private symbols.

I'm not trying to claim that ReactOS is clean, I have no skin in the game from a project or user perspective. For all I know it might have lifted significant portions of its code from stolen source code or the WRK (which isn't stolen in so much as used without permission, which I'd regard as a totally different thing). I do however take exception to the typical software engineer's view there are somethings which cannot be reverse engineered into a almost similar form.

As to why ReactOS is stuck in the early 2000s, it could be because of all the source code which was stolen and put wholesale into the project. Although if that was the case I'd have expect MS would have sued the living shit of the project by now. It could also be because Windows was and is a very complex OS with many layers which if you're trying to re-implement with a team of 10s to 100s versus 1000s it's going to take a lot of time. It's seems unlikely that the project would spend the millions of man hours to create the abomination that is UWP.

Perhaps the best way to determine if ReactOS is unclean is for MS to open source the Windows Kernel, hell why would you even need ReactOS then :-)

edit: Cleanup.


That sounds reasonable (symbols inadvertently leaked). This would be a plausible explanation for at least some of the naming similarities.



The above link points to: “OffensiveCon19 - Alex Ionescu - Keynote - Reversing Without Reversing” and to 9th minute of the talk where it’s very relevant to the current discussion.


Someone pointed to that earlier. Great talk about how he gathers information to write his books.


See again what I quoted up in the thread related to the NSAKEY:

“Windows NT 4 Service Pack 5 (which had been released unstripped of its symbolic debugging data)”


ReactOS goes greater lentgth to preserve backwards compatibility.


Often hotfixes came with private symbols, Microsoft has traditionally been very slack on this.

I’ve seen private symbols for sql server with the guid to switch editions published on the public symbol server for at least 6 months before they were pulled.

Full releases and service packs typically are stripped very well but if you are saying that no private symbols have been published to the public symbol servers then you are incorrect.

The only product that has been effective at stripping symbols traditionally has been office, they were always stripped if you could even get hold of them which was unlikely.

Don’t forget also you could download the checked windows builds which were very open.


> Then if reversing the kernel is so doable using legitimate means, why ReactOS is still largely stuck in the early 2000’s, coincidentally where the major leaks happened?

Because they don't care or need about the newer MS stuff and also don't have the resources either.

Also they can exploit Microsoft good record of backward compatibility, once you have a good enough lower API compatibility, you can just install a lot of newer MS tech directly on top of it.


> Sure they don't release private symbols intentionally but they have done in the past accidentally.

Even if that is the case, it's an incredibly poor idea to use them, so that the code ends up with spurious similarities in spite of being (otherwise) cleanly developed.


Are you arguing that MS mistakenly releasing a version of Windows with debug symbols didn't happen or that it constitutes a leak and isn't fair game?

Because the former is fairly well documented and the latter doesn't seem right in the context of this discussion. If MS themselves messed up and published the symbols through an official channel that's fair game IMO. Although obviously IANAL etc... I'm talking from an ethical perspective, not a legal one.

I don't know much about ReactOS or the NT kernel but we have this type of controversy regularly in the emulation scene and while sometimes it's true that people reuse docs they shouldn't have, a lot of the time people underestimate the skill and cunning of reverse-engineers to figure out how things work without having access to any restricted information.


Kind of late to the game here, but MS could ship actual source code and copying that code would still be copyright infringement. Reverse engineering for inter-operability is legal in many places, but copying implementations is not. Even having seen the code, you would have to reimplement it in a way that worked equivalently, but was different. The OP is claiming that even things like macros have the same implementation and names (i.e. code that is never exposed publicly in any way). Even if you could deduce this from debug symbols (which would be quite tricky, but probably not impossible), you've got to find another way to do that work.

I don't really subscribe to a belief of absolute morality, but in the context of the discussion, I think that no matter how you got access to that code, if you say that you are reverse engineering it, then copying an implementation is not doing what you are saying you are doing (as well as being copyright infringement).


> Are you arguing that MS mistakenly releasing a version of Windows with debug symbols didn't happen or that it constitutes a leak and isn't fair game?

I think he's saying that even having access to leaked or accidentally released originals is not implicit permission to use it freely. Otherwise any piece of software that was ever legitimately released would be fair game, just throw it at a decompiler and profit.

If you're making a clean room design having so many similarities to the original is unlikely to happen accidentally.

Anybody implementing a clean room design should theoretically have no prior knowledge of the original's inner workings. The specs are written by one person, checked to not include any of the original material by a second one, before being passed to a third to be implemented.

From far enough a piece of wire and an isolation transformer do the same thing. The secret sauce is in that isolation, you can't just shunt it and pretend it's the same.

dingby 16 days ago [flagged]

You chuckleheads “released” pretty much everything in the ASSERT statements of checked builds until Windows Vista.


Personal attacks and name-calling, as here and in your previous two comments, are not ok on HN. We ban accounts that do that. It's great if you're providing correct information, but please edit out the swipes.

https://news.ycombinator.com/newsguidelines.html


One small correction. On quora, you state that there is no such thing as public documentation for the NT kernel internals.

There IS public documentation for the NT kernel internals. It is called Windows Internals Book (https://amzn.to/2xCQla5) and has survived its’ 7th Edition


And one of the authors of said book is also quite a prominent member of the ReactOS team.


Did he work for Microsoft though? If yes, I wonder what kind of interaction between him and Mark Russinovich.


No, he did not. Mark Russinovich also started to work on Microsoft only recently.


Mark Mussinovich started working at Microsoft in 2006 according to Wikipedia https://en.wikipedia.org/wiki/Mark_Russinovich


Yep, you are right, https://www.linkedin.com/in/markrussinovich/

But still in 2006 ReactOS already had its 0.3.0 version and 8 years of existence ^) And book Microsoft Windows Internals, Fourth Edition has already been released in 2004


His company got acquired by Microsoft


I cannot edit my comment for some reason, sorry for the typos. About my colleague’s hack don’t get too excited, this is ancient history but it was at the center of a public debate at the time. Some 3rd parties had an ask, the company said it was difficult, this was very true but someone inside came up with a very clever hack that satisfied all parties, the issue was settled and everybody lived happily ever after. It’s just that the nature of the hack relied on very specific implementation details and it’s highly improbably that another implementation would need a hack to begin with, let alone the same hack. It’s one of the many coincidences that I believe will be hard to explain, shall someone really ask.


Most of the kernel code in Reactos is from far before Alex started working at Microsoft. Also the official Microsoft documentation is of course far from the only ‘Windows Internals’ documentation.


Microsoft has leaked tons of private variable names publicly though. See the whole _NSAKEY thing for one example.

https://en.wikipedia.org/wiki/NSAKEY


This is news?

Of course it’s not a proper clean-room copy. This is immediately clear to anyone who’s worked on NTOS or who has source code license.

It been obvious for years. The only possible conclusion is that MSFT doesn’t care.


Anyone spending time giving detailed explanations regarding manual transmissions and engines in general deserves a follow :)


[flagged]


Thanks for the personal attack lol. I watched and did not learn much, except that he’s a quite decent orator. I think what he describes is the process he uses to write his books.


In that talk he literary says how he got names of files and variables.


Why wouldn't you reply to ReactOS devs here pointing that what you are saying is clearly wrong?


What would he say other than what he has already said? The ReactOS devs are not addressing the substantive points. No point in him repeating himself.


Em, for example how the book linked here several times is not documentation on kernel internals?


It’s not documentation no. Some smart people spent a lot of time observing and disassembling and explained a number of things but major aspects are entirely missing, and not all of what is written is accurate. In any case this definitely is not documentation, and certainly not enough information to write not only a compatible implementation, but a compatible implementation with very similar-looking source code.


I hadn't heard of ReactOS but a quick glance at it's kernel source on GitHub and it's OBVIOUSLY straight from Windows.

Not that I work for MS but I used to work on NT graphics drivers and had a copy of the leaked NT kernel source for a while (strictly for personal interest!). This so very like what I remember.

IMHO such blatant theft shouldn't really go unpunished.


I kind of doubt this is the case, because more than 10 years ago they had accusations of this sort and undertook an extensive audit and cleanroom rewrite: https://www.reactos.org/reset-reboot-restart-legal-issues-an...


Microsoft complaining about ripped off source code?

You are the blackest pot in the history of computer software.

https://www.geek.com/news/microsoft-uses-open-source-code-54...

https://lwn.net/Articles/245805/


It's BSD code in both examples. Given that the BSD license explicitly permits the sort of use in question, ripped off seems inaccurate.

Not that microsoft has never crossed any lines - even the naming of the Windows operating system seems morally dubious - but in this case I don't see any issue with what they've done.


It seems like you're personally offended by the project. If some of the code that got presumably stolen is yours I can definitely understand.

Looking at how slowly that project is moving though I think your conclusion is right: They don't care. In some other comment you mentioned how the project seems to be stuck in the 2000/XP area. How relevant does that make this project today? But whether it's based on stolen code or not, it seems like the right choice to focus on getting that completed before moving on to everything that came after. And after all, Windows isn't just the kernel. They have to implement a lot of user space too, because software out there might assume it's available and behaving in very certain ways. Just an explorer clone sounds like a mammoth task on its own.

I still find the project very interesting and overly ambitious, even if code was stolen. I think at their current pace, reactos won't become usable before operating systems became largely irrelevant and everything runs in the cloud. Everyone still working on it today is probably just doing it for the challenge.


Uh-oh. Someone didn't check with MS LCA before posting this. Bad move.


I expressed my personal opinion, here and on Quora. I think I’ll be fine.


Can you please explain what is MS LCA?


Legal & Corporate Affairs


non existent - they are called by a different name in MS (and also have clear guidance for things like this, which was not broken)


Used to be LCA. Now CELA. I left in 2009.


He needs to defend himself in internet court, employment be damned. /s


I find an opinion hard to credit when exact proof is sitting their on an open source repo everyone has access to and a closed source repo only you have access to.

You make a theoretically compelling argument like the "hack" you describe above and then leave out the only part that really matters where you prove it. It is entirely likely that you are simultaneously entirely correct and entirely unable to prove it due to company policy but this leads us to the next logical question.

If you knew you couldn't prove it why did you open your mouth? It seems the only result will be negative PR.


It appears you did not read what I wrote. The similarities are out there for anyone to see, there is plenty of leaked Windows code floating around including on GitHub, and I did not need anything else to form my opinion.


This was flagged for some reason. Please don't flag comments that don't violate community standards just because you don't like the author.

I have no intention of reading through millions of lines of code from a legally questionable source to make your point for you.

How about you do so and post an in depth analysis?


Do you think senior management at MS believes your comments are a good idea or at best benign? Or do you think all the details of your comments have just not yet fully reached VP level, legal, and PR?

I'm thinking they'd disapprove. That's just a guess, maybe they think it's fine and great. However, it may be worth at least considering another possibility.

Does your company want random technical employees, making non-technical statements, about how people are "ripping off" MS?

It's your job to raise the flag internally and help people understand up and down the chain, and across functional lines the context and perceived severity of the issue.

It's your job to write a blog post about how great your containers are architected. Not be a lone wolf, self appointed spokesperson using inflammatory language on issues with potential legal implications.

Please don't take this comment in the vein of you're just an engineer shut up and code, in fact you have a critical role as someone with domain expertise.

The point is when you stumble onto a high visibility cross-functional issue, I've found on sensitive topics many organizations seem to appreciate when someone reacts by coordinating, discussing, and facilitating a cross functional, unified response with one designated voice, that acts in the best interests the company.


Why are you policing other company's employees and presuming to know what their job is?


First off I'm not policing anything, because, how would I have the authority to do that?

Secondly you object to me presuming I know what his responsibilities are. I don't claim to know what he codes. However I'm able to make a pretty good guess about what the scope of a MSFT senior developers job is and this kind of thing is not even close, unless it's a special case worked out in advance with others. It wasn't worked out in advance in this case because he says that he made the comments before getting feedback from more senior people on his team.

Finally I assume some of the down votes have mistakenly conflated my comments with trying to hide information behind corporate walls.

I advocate hiding nothing - and fully support the new generation of companies who believe in transparency and ethics.

Being transparent and ethical has nothing to do whatsoever with letting developers try and make legal decisions when it's not their area of expertise. It also doesn't mean that a company shouldn't work together across departments to try to decide the right thing to do.

Good management and coordination between roles don't preclude transparency or doing the right thing in anyway shape or form .


This was a fairly pedantic post but I'll reply anyways.

> First off I'm not policing anything, because, how would I have the authority to do that?

Maybe policing isn't the right word, but your earlier post was fairly condescending to axelriet. "I think they'd disapprove... It's your job to do this... It's your job to do that..."


It's subjective, but let's be real, you're probably right.

Moreover, I don't believe in such a philosophy between any two people, so to the extent it is I regret the tone of the comment.

If it makes any difference, the true source of the tone was based on thinking similar to, holy crap that person is backing up into a stove someone speak loudly and quickly... There is no evidence for you to believe that, but I hope you will, somehow, decide the intent was nothing more.


My opinion about ReactOS has nothing to do with my job or my employer, you are trying to make this into something it is not.


Your opinion has a huge amount to do with your job and your employer.

Appeal to authority may be a classical logical fallacy, but it's a highly effective one and is used by many people to infer the merit of an opinion rightly or wrongly so. Yeah, I know it's weird, that people would associate a comment one person posts with the voice of a trillion dollar company. Just happens sometimes.

Moreover I have nothing to gain by trying to "make this into" anything.

Believe it or not, I actually hoped it could be of some help to you as another data point - as far as all the things you take into consideration when choosing how to exercise your bill of rights free speech, which has nothing to do with your corporate free-speech. The latter of which unfortunately has cost many people dearly.

With respect to Microsoft, I'm certainly not making any comment about them as a company one way or another. That wasn't the point because, this concept applies to most large similar tech companies.

Things do change rapidly at such companies. Fwiw, I have done your job, your bosses job, been the janitor, and a few other things there. Things may be different now, but i'm not making the most possible uninformed guesses just to mess with you.

I sincerely hope the truth is known and that there are no negative repercussions for you personally.


I’m not speaking in the name of the company: I expressed my personal opinion about ReactOS and wrote back in 2017 that I think it’s a ripoff, after looking at their code. I’m not out there to destroy ReactOS, but if you ask me today what I think about it, I’ll give you the same answer.


Axel,

Look at my HN profile and contact me, so I can send you an email and ask you something. Or, put a way to contact you in your HN profile.


What is the point of comments like this besides aiming to seem smarter than the original poster? If you get your way then you would stifle open discussion and information.


I like this honesty and transparency, employers such as Microsoft encouraging it's employees to act in this way is a good thing.

Shutting down discussion behind lawyers and bland public statements should be discouraged.


We like it. But it can be actively harmful to the interests of the company. And its impossible for us to have the facts required to know if this is harmful. That's up to the company. So if they decide to fire employees who are spending their free time harming the company, we can't really get upset about that.

Its simply impossible to say, "no but this openness is good for the company" in an informed way. But we can certainly hope the company comes to this conclusion and encourages openness.


Collaborating across departments with people from different functional areas has absolutely nothing to do with dishonesty or lack of transparency.

If you have good people in those positions in your organization, they will come together and make good decisions.

If you don't have the right people in those positions, then that's not going to be a good thing irrespective of my advice.

Transparency doesn't require anarchy and anarchy doesn't buy transparency.


There are other jobs, would you really want to work some where that makes you afraid to have an opinion?


In 2006 ReactOS halted all developments because of accusations like this. After a year long effort there was found to be no use of Microsoft source code in ReactOS. (https://www.reactos.org/reset-reboot-restart-legal-issues-an...)

From what I understand a lot of the kernel work was done by clean-room reverse engineering. One person decompiles and documents the system and another re-implements given the documentation. I can easily see how this would result in identical structures and naming.


Is that really clean-room, though? (Note that "clean-room" is not a term in US copyright law, it's just a technique that certain people hope results in a work that is not a derivative of whatever they're reimplementing.)

If I learn a pop song, and sing it and accompany myself on the piano, and someone else transcribes it, then two things are true: first, of course it will result in identical melodies and chord progressions, and second, of course (it seems to me) this is a derivative work and there's no way you can say its copyright is independent. On the other hand, if I learn a pop song, and then teach the fundamentals of songwriting to someone, and they write a song with a I-V-vi-IV progression with angsty lyrics about an estranged lover, no matter how much it resembles existing pop songs in intent, there's a much stronger argument for independent invention.

A "clean-room" approach to determine a specification where none exists is sound - e.g., if you're developing a driver for a piece of hardware which only has a proprietary driver, you can run the driver under instrumentation and see what it does, and then write your own driver that twiddles hardware in the same way but takes its own approach to implementation. A "clean-room" approach to determine internals feels less sound to me: if you're disassembling the code to determine structure layout and naming, aren't you really just transforming the code into an overly-detailed natural language representation, then back into code?

(To be clear, I don't know if this is what ReactOS does/did, I just think that this "clean-room" defense doesn't hold super strongly, or in other words, that if this is what ReactOS does/did, it can be true both that ReactOS believes they did a clean-room reimplementation and that it's an infringement of NT's copyrights, despite a lot of hard work by the ReactOS devs.)


>> From what I understand a lot of the kernel work was done by clean-room reverse engineering. One person decompiles and documents the system and another re-implements given the documentation.

> Is that really clean-room, though?

That is precisely the technique that Compaq used to reverse engineer the IBM PC BIOS. I'm sure it can be done in a legally risky way, but the fundamental technique has been quite well tested, legally speaking.


I don’t remember the details of the BIOS case. A BIOS responds kind of like hardware. You put values in a register, trigger an interrupt, and look at the result in another register. As I recall, the clean room implementation replicated the same behavior, which is permitted. But documenting the actual structure of the code, including variable names, and copying that is not clean room reverse engineering from the point of view of copyright law.


> If I learn a pop song, and sing it and accompany myself on the piano, and someone else transcribes it,

I hate bad analogies. A bad analogy just confuses the issue.

There's one and exactly one song called "Billie Jean", which was written, composed, and performed by Michael Jackson.

There are however a thousand different works relating the history of Michael Jackson's work with Quincy Jones in producing the song "Billie Jean" for the album Thriller.

So is there one and exactly one implementation of a Windows NT kernel, or is there a possibility there could be many different types of Windows NT kernel implementations?


So, if I sing a song with lyrics "Billie Jean is not my lover" with melody re-re-do-la-re-re-do-la, is it a different song because it was recorded by me? It is pretty clear in copyright law that my recording would infringe Michael Jackson's song.

What does "a Windows NT kernel" mean? Exposes the same interface? (Does Linux + WINE count?) Operates internally in the same way?

Maybe a better example is UNIX. Linux offers the same conceptual interface, but is neither binary-compatible with UNIX nor is implemented the same way. It is not UNIX, and does not infringe. NT, FreeBSD, and Solaris all have / had compatibility interfaces for Linux that are binary-compatible but wildly different internally. They also do not infringe. And no "clean-room" work was necessary to reach this state.


> So, if I sing a song with lyrics "Billie Jean is not my lover"...

Please stop talking about music, because it just further confuses your thinking about software copyright.

> What does "a Windows NT kernel" mean?

I mean the plain understanding of "Windows NT Kernel". See wikipedia:

https://en.wikipedia.org/wiki/Architecture_of_Windows_NT https://en.wikipedia.org/wiki/Kernel_(operating_system)


Clean room re-implementations have stood up to many court challenges through the years, and are interoperability is explicitly permitted under things like the DMCA.


Yes, many clean-room reimplementations have stood up to court challenges. But many non-clean-room reimplementations have stood up, too. My argument is that it's neither necessary nor sufficient, just loosely correlated, and clean-room is not inherently a defense. And in particular, my argument is that it matters whether the communication between the dirty and clean rooms is a specification of an interface, or a description of a reverse-engineered implementation.

And yes, interoperability is permitted - that's why I'm saying it seems fine to clean-room reimplement a driver to communicate with a specific device. You do not need to have the same internal structures and the same function names to achieve interoperability, and I think making that argument that the DMCA allows you to have the same internal code layout will be hard.


> You do not need to have the same internal structures and the same function names to achieve interoperability

That might be the case for interoperability with a hardware device, but you probably do to make an operating system that's compatible with Windows software and drivers, considering how historically brazen developers are at abusing internal details of Windows. Consider virus scanners that happily patch internal data structures like the SSDT, or other programs that detour a bunch of API functions, then expect those functions to be called in a specific order when a different API function is called, or programs that just reach into stack frames they don't control and pluck out values[1]. (There are some other good ones in that link.) Windows takes pains to be compatible with insane programs like this, so you could argue that copying weird implementation details like this is necessary for interoperability.

[1]: https://devblogs.microsoft.com/oldnewthing/20031223-00/?p=41...


Considering IBM couldn't stop Compaq after taking some pretty good steps, it has been legal for a while. Without clean room re-implementations we would have no PC clone industry.


Reverse engineering is usually considered a violation of proprietary software IP right (not sure the exact legal term)? I know at every company I've ever worked at I've been told to never under any circumstances attempt to decompile or reverse engineer another company's proprietary software product. Maybe that's just legal CYA but decompiling+documenting then re-implementing sure sounds like stealing to me...


In the United States, clean room reverse engineering is legal for interoperability purposes. In general reverse engineering is legal unless there is a contract involved prohibiting it.

Unless you’re reusing the original source code, there’s no copyright violation per se, simply functionality reimplemented in a compatible manner. The lengths that ReactOS goes to might actually be overkill, but no one can credibly claim the project has actually stolen anyone’s code.


Does a software license in this case count as a contract?

Moreover, interoperability is not the same as distribution no? Honestly ReactOS's lengths do not seem like overkill to me, quite the contrary. Even re-implementing an API is legally suspect these days (see: Oracle vs. Google re: Android). Copying the internals of a piece of software by having one person explain verbally exactly how it works to another person who translates that to code feels a lot more like stealing than that.


A EULA might but I’m not positive that is the case. That said, I would expect a business of any size to cover their ass and not bother without a compelling reason.

That said, look at the steps involved.

Person A had the original software in binary form, runs it through a decompiler. The decompiler is not going to reproduce the code precisely, but it will be enough for documentation purposes. They document the functionality in a separate document describing what is happening when the software is run.

Person B receives this document and writes their own code according to what it says. They need not have ever even used the original software or signed a EULA, and then Person A or C or whoever can go on to test compatibility. It is a lot of effort, but the end result is code where no one involved violates any laws. It’s interoperable, and because the code is original, Person B or the Project he works for can claim copyright and license it out on their own terms, in the case of ReactOS, it is an open source license.

Re: Oracle vs Google, this case does make the whole project potentially suspect these days, but if APIs are not copyrightable, then ReactOS should be firmly on the right side of the law. If APIs are copyrightable, then we can expect the Wine project to be taken down alongside ReactOS. More reason to hope Google prevails entirely, regardless of our own personal opinions of them.


Yes, EULA is a contract and has been successfully tested in court against reverse engineering: https://www.eff.org/cases/blizzard-v-bnetd


For the contract portion of the EULA to be enforceable, you have to get the user to explicitly and willingly agree to it.

Tricking the user with a shrink-wrap license ("by opening this package you have agreed...") would not hold in court.

Asking users to click "I Agree" in installers also has dubious value due to the widespread practice of clicking these buttons to proceed without really reading the contract; their true intention was proceeding to the next question rather than entering any agreement.

In other words: if most reasonable persons wouldn't expect to be actually bound to the conditions listed in the contract, then Microsoft cannot effectively enforce them in a court.


> Tricking the user with a shrink-wrap license ("by opening this package you have agreed...") would not hold in court.

Your claims are legally dubious at best. https://en.m.wikipedia.org/wiki/Shrink_wrap_contract

“One line of cases follows ProCD v. Zeidenberg which held such contracts enforceable (see, e.g., Bowers v. Baystate Technologies[2]) and the other follows Klocek v. Gateway, Inc., which found the contracts at hand unenforceable (e.g., Specht v. Netscape Communications Corp.[3]), but did not comment on shrink wrap contracts as a whole.”


Couldn't that trivially be defeated by having that part of the work done somewhere that doesn't accept such shrink wrap contracts? Alternatively even just a different person than actually paid for it?

If I receive a box in the mail or an exe in an email along with a request to document it to what degree am I party to any contract with the developer? I received after all nothing from them.


>Unless you’re reusing the original source code, there’s no copyright violation per se

Dunno about that one, chief. Copying non-literal elements of code can still be infringing.


You seem to have internalized ethics designed to maximize profit not happiness or utility.

Deliberate incompatibility is a lever that allows a degree of control that allows one to extract money both from end users and from companies that serve end users.

Having the ability to reverse engineer a compatible solution is a check on the maximum amount of rent seeking one is able to do before a competitor provides a more open or at least cheaper alternative that doesn't force one to entirely rip out existing components.

This is only like stealing if you believe a company has an inherent right to make a maximum profit without interference from competitors which is an interesting moral stance to take.


Reverse engineering is generally allowed. Trade secrets don't have a lot of protections. The reverse engineering + reimplementing route has precedent in the US, the Compaq vs IBM BIOS case.

Companies mostly have these rules to guard against copyright and patent lawsuits because there is no presumption of innocence beyond reasonable doubt in these in the US.


ReactOS underwent a big audit years ago because people kept saying things like this, and AFAIK it's over and nothing exciting turned up.

[1] - https://reactos.org/wiki/Audit


The author of post, Axel Rietschin at Microsoft, should remove this post and publicly apologize for his baseless defamation of ReactOS ("like a baby mosquito on the back of an giant elephant") if he cannot give authoritative evidence to back his claims. There may be copyright issues in some subroutines, but calling the entire project "a ripoff of the Windows Research Kernel" is an extraordinary, arrogant claim. If you have extraordinary evidence, fair enough, otherwise you should STFU.

Update: my original comment was refering to the author's original answer, which didn't give a single symbol name. But apparently, the author registered an HN comment and start giving more details than what has been said in the original answer, I need to stick to the facts so I'd say the credibility of the accusation is no longer 100% "baseless". Let's see what is going to be the evidence.


It’s my personal opinion and it’s based on my lecture of ReactOS code at the time I downloaded it (circa when I wrote my reply on Quora). I think anyone who can read C code can reach similar conclusions just by eyeballing the code in ReactOS and the code from the leaked Research Kernel, that can be found for example on GitHub. As a matter fact, someone on this thread did just that and it did not take that person very long (just minutes) to find some of the similarities that I, for one, qualify as troubling. It should not be a problem for anyone to repeat the experiment, all code is available for download. So why not go ahead and see that code with your own eyes? I think it’s highly improbable that a reimplementation of that magnitude came so close from the original in so many respects. I’ve been conducting interviews for years and screened some of the best and most promising future engineers coming from the best universities and I always ask the same coding questions. I was given many decent and correct answers, but never the actual implementation was similar to the level I can observe between ReactOS and the leaked Microsoft code. In fact every candidate wrote very different answers to the very same question, some even had very different approaches to the resolution of the same problems. I cannot explain how a team of people on a hyper-complex open source project can write nearly the same code - the source code looks very similar down to peculiar formatting idiosyncrasies - as a separate team of developers of the original closed-source project, just from observing the external behavior and/or reading the documentation, if available at all. I think this is impossible.


>and it’s based on my lecture of ReactOS code at the time I downloaded it

It would be interesting to see if this statement comes back to bite you in court if the opposite accusation is ever made. That Windows kernel developers have been stealing from GPL code :)


Man, I thought the exact same thing. I must have stolen it from you. Sorry!

In all seriousness is this normal? I write and work on kernel drivers for the company I work for and have always gone out of my way to not view or interact with any code that remotely relates to the work I do (out side of what I coded). Our competitor even released their version of a block device snapshot driver and while I would be tempted to see how they did things I have avoided it every step of the way.


So, if a Microsoft kernel developer doesn't follow the practice of not reading kernel code with an incompatible license, does that mean we can assume they don't follow it generally? Can we assume they take code from the Linux kernel, violating the GPL?


Possible.


Doesn't GPL actually offer a person a freedom to study the code? Isn't it a common phenomena of rewriting GPL Linux Driver into BSD Licensed driver which will be put into mainline *BSD?

I know the case of OpenBSD 'stealing' GPL code from broadcom linux driver, but it was done by a naive kernel developer who copy-pasted the code and the maintainer who didn't bother to check the authenticity of the code. But AFAIK no GPL driver writer has a problem with the driver being rewritten into BSD licence.


GPL is absolutely incompatible with BSD licensing. The license does not allow for one to "rewrite" the same code into a BSD licensed driver.

Of course one could clean room reverse engineer the GPL driver and then give the spec to someone else to implement and create a BSD driver, that would probably be ok.


Not exactly incompatible, as you can license something under both of them at once. But yes, you cannot relicense it without permission from the contributors; this is true for pretty much any license.

Right, of course the rights holder can license under anything they want. But no one else can take GPL code and relicense it under BSD. The same is not true the other way -- anyone can use BSD code.

MS Windows kernel code is far from being BSD licensed though.


I presume you quit working on Windows after you read the ReactOS source code, offered to you under the GPL license only? Or is Microsoft planning to release it under a compatible license?


Plot twist: Microsoft employee goes out of his way to force Microsoft to open-source Windows


Your interview analogy is a bad one. Because they are not trying to make their solutions as similar as possible to each other.

If you had interviewee A write some piece of code, and then said to interviewee B "I want you to solve this problem as close as possible as the person before you did it, here is their executable", you'd be quite surprised what they could do. Especially if interviewee B is Alex Ionescu.


I don't suppose you ever compared parts of the tree that weren't present in the various sort-of-public NT source leaks or research kernel, to see if idiosyncrasies there matched as well?


> It’s my personal opinion

Others in the thread have pointed out several verifiably false fact claims in it, so it's both derogatory and at least in part factually false, not a mere matter of subjective opinion.


Saying "like a baby mosquito on the back of an giant elephant" is not a defamation. It's a metaphor. He's saying that the project doesn't really threaten Microsoft so they'll probably just ignore it.


>baseless defamation of ReactOS ("like a baby mosquito on the back of an giant elephant")

If only he had called it a cancer. That seems to work well generally.


The infamous "cancer" analogy was referring to the copyleft-licensing, not the actual system per se.

Saying ReactOS infringes 200+ Microsoft patents and threatening for lawsuits to developers and all users is a more effective FUD approach to me, and may even be factually true. The entire project of Linux kernel was running under this threat for 10+ years until ~2005 (?) when an agreement was established.


AFAIK Software patents aren't enforceable outside of the US+Japan though.


That would be enough to get an injunction and prohibit the software from being distributed in the US, which would deny a very important market for the developers.


Indeed, but as the world becomes less and less centered around the US, this effect will probably disappear soon: if a product is originally developed for China, not being able to sell it in the US isn't an issue.


> if a product is originally developed for China, not being able to sell it in the US isn't an issue.

If it's a commercial product or an academic research project under national grant, it may be the case.

But for free and open source project, banning U.S. developers from participating is (still) suicidal.


Don't we all think if the accusations were true, Microsoft would have aggressively persued legal action? Remember that guy they sued into oblivion and maybe prison over "rescue discs" for old PCs? Microsoft it notoriously sue happy. Why not here?


Because reactos is irrelevant to Microsoft?

The project has been around for 20 years, it's not likely to cause MSFT to lose any revenue any time soon, and there is no large corporation to extract money from in a lawsuit.

It's basically the same reason they don't go after people who have a pirated version of Windows and just show them a "this is fake" warning.


Also, attempting to take it down would surely cause reputation damage to Microsoft. The "old" Microsoft is somewhat infamous. Going after the small guy would not help their image.


I hate to say this: the guy who printed the "rescue disks" also decided to put the standard Windows Install Disk graphics on them. He got sued over the whole thing, but they look the same, and he didn't pay is what got him.


Because Nadella is trying hard to win the hearts of developers and attacking an open source project now like they did with Linux would backfire on them.


Because what Microsoft did instead was offer Alex Ionescu a job.

Interesting, but I’d really like it if they actually pointed out some examples. There may in fact be stolen source code or at least it is possible some stolen source code was referenced in ReactOS - there’s virtually no way to prove that it didn’t happen - but one human’s observation could be very flawed and represent a view extrapolated from a small subset of stolen code. How much of the code could they have possibly read?

It’s also worth noting that large portions of ReactOS come from Wine and I’m sure those portions have plenty of internal NT structures too.

The bits about NT not being documented are bordering silly, though. The lack of documentation has certainly never stopped anyone. Stardock for example does some very interesting and very undocumented things, as do many (most?) DRM and obfuscation engines.

Personally, I think that if ReactOS has really stolen substantial amounts of code, then it genuinely didn’t pay off and probably was not that useful.


I was curious, and since the author of this answer provided links to what they called leaks [1], it's not difficult to download both the leak and ReactOS source [2] and look for similarities. So, out of curiosity, that's what I did.

I started poking at some source files at random in ntoskernel, and it only took a minute to get some "hits" from the ReactOS source:

    $ rg InitializeContextThread
    ./ntoskrnl/ke/thrdobj.c:872:        KiInitializeContextThread(Thread,
    ./ntoskrnl/ke/powerpc/thrdini.c:53:KiInitializeContextThread(IN PKTHREAD Thread,
    ./ntoskrnl/ke/i386/thrdini.c:92:KiInitializeContextThread(IN PKTHREAD Thread,
    ./ntoskrnl/ke/amd64/thrdini.c:36:KiInitializeContextThread(IN PKTHREAD Thread,
    ./ntoskrnl/ke/arm/thrdini.c:53:KiInitializeContextThread(IN PKTHREAD Thread,
    ./ntoskrnl/include/internal/ke.h:464:KiInitializeContextThread(

    $ rg NpxFrame
    ./ntoskrnl/ke/i386/v86vdm.c:473:    PFX_SAVE_AREA NpxFrame;
    ./ntoskrnl/ke/i386/v86vdm.c:480:    NpxFrame = &StackFrame->NpxArea;
    ./ntoskrnl/ke/i386/v86vdm.c:481:    ASSERT((ULONG_PTR)NpxFrame % 16 == 0);
    ./ntoskrnl/ke/i386/v86vdm.c:485:    RtlCopyMemory(KiGetThreadNpxArea(Thread), NpxFrame, sizeof(FX_SAVE_AREA));
    ./ntoskrnl/ke/i386/v86vdm.c:510:    PFX_SAVE_AREA NpxFrame = &StackFrame->NpxArea;
    ./ntoskrnl/ke/i386/v86vdm.c:512:    ASSERT((ULONG_PTR)NpxFrame % 16 == 0);
    ./ntoskrnl/ke/i386/v86vdm.c:551:    RtlCopyMemory(NpxFrame, V86Frame->ThreadStack, sizeof(FX_SAVE_AREA));
Now that said, I have no idea what I'm looking for, or if this is significant. But since we have the source for both, it seems we can do better than speculate w/r/t how much code is similar between them. I know that does not prove copying but I think the author's point can't be blindly dismissed.

[1] https://github.com/Zer0Mem0ry/ntoskrnl

[2] https://sourceforge.net/projects/reactos/files/ReactOS/


I do not wish to view the leaked copies of Windows source code, unfortunately. It does have potential legal ramifications, in the United States. If someone who doesn't care can attempt to audit it, that would be absolutely great.

That said, KiInitializeContextThread isn't 'secret' enough to not appear, it is a well-known bit of internals. Here you can find some docs about it on Microsoft's own website, albeit I have no idea if this is present anywhere on there anymore.

https://web.archive.org/web/20151214014944/http://www.micros...


If Microsoft cared, they wouldn’t be hosting that on their very own servers.

GitHub belongs to Microsoft. They could pull that down at any time just because they feel like it. But they haven’t. That’s a pretty solid sign that there’s not any legal consequence to viewing it.


Well, my paranoid 99% says that they could keep it online on purpose so that all people viewing it fall in a legal disadvantage position. Removing that code would achieve nothing in secrecy today compared to the legal leverage they can get tomorrow by keeping it online.


For what it's worth, it's not just Microsoft who care. You're not allowed[1] to contribute to Wine if you've ever seen the leaked Windows source code. I think it's weird that they consider your mind to be "tainted" forever, but whatever. I guess they want to be cautious.

(You're also not allowed to contribute to the Wine CRT if you've ever seen the CRT source that's legally distributed with the Windows SDK.)

[1]: https://wiki.winehq.org/Developer_FAQ#Who_can.27t_contribute...


Just going to throw this out there, if Microsoft cared that anyone viewed it, wouldn't they have taken it down by now?


I don't think Microsoft cares that much, but I'm sure they view it for what it is: illegal, unauthorized usage of their intellectual property. (I'll hold off on my personal views of 'intellectual property' and whatnot; I'm not a lawyer, and those opinions are hardly relevant to the reality here.)

Still, even if Microsoft doesn't care, Wine Project and ReactOS care. My employer might care. There are unfortunately legal implications that exist, and if people want to play by the books they have to at least do the basic due diligence.

If you genuinely just don't care, you can lie to them and claim you never saw the original code, and attempt to cloak any code theft to make it indistinguishable from clean-room reverse engineering. Nothing stops you from doing that. Hell, the trouble is that it really can't be proven for sure. Even holding the positions I do on forums wouldn't prove my own innocence in such a situation, though maybe it helps build a case.


What stops someone from viewing that repo over Tor, on a different git server, or at a public library, unsecured WiFi network, etc and being considered tainted? At what point is someone considered untainted?

This seems like a grey area that slows the pace of R&D in countries that believe code is copyrightable & patentable (the US and select Western countries) compared to say New Zealand & China, where these aren't an issue.


Where did you get that idea about New Zealand copyright? Code is copyrightable here, see e.g. [1] or [2].

And it's not about belief, but definition. Copyright and patents are human inventions defined by countries' laws (which is why different countries have different rules: different definitions). In NZ, as in many countries, "Computer software follows the rules for literary work" ([2] again).

[1] http://www.burgess.co.nz/copyright-ownership-and-software-de...

[2] https://www.consumer.org.nz/articles/copyright-law


New Zealand abolished patents for software, that is what I was thinking of: https://www.washingtonpost.com/news/the-switch/wp/2013/08/29...


It was more that we clarified what was and wasn't patentable, as I don't think software was exactly patentable before that either; its patentability and the ability to enforce said patents were ambiguous. Patents on software are not quite completely banned, either, as, from your link's linked source, "Processes will still be patentable if the computer program is merely a way of implementing a patentable process."


I tried googling for KiInitializeContextThread and there are lot of search results for this name. For example, a presentation in Chinese dated 2005 year described process creating and mentions this function: [1]

The sequence of events that happens inside KeInitThread() can be obtained from reverse engineering. If you compare the code of this function in WRK [2] and in Reactos [3] code, you'll see that while they do similar things, the code is not the same.

Regarding NpxFrame, I don't know why this name is used in KiExitV86Mode, but if you compare functions KiInitializeContextThread in WRK and in Reactos, you'll see that in WRK the variable is named NpxFrame [4], and in Reactos a similar variable is named FxSaveArea (which matches structure FX_SAVE_AREA). The variable named NpxFrame in Reactos is used in other function, KiExitV86Mode.

[1] https://slideplayer.com/slide/6561298/

[2] https://github.com/Zer0Mem0ry/ntoskrnl/blob/master/Ke/thredo...

[3] https://github.com/reactos/reactos/blob/master/ntoskrnl/ke/t...

[4] https://github.com/Zer0Mem0ry/ntoskrnl/blob/a1eded2d8efb0716...

[5] https://github.com/reactos/reactos/blob/893a3c9d030fd8b078cb...


I'm confused about those examples. Both symbols seem to be referenced in quite a few web resources that aren't dumps of that research kernel and either github search is a bit broken or I can't seem to find many of those in your [1] repository.


I have a book on my shelf somewhere about NT internal APIs; I forget the title or publisher as ive not looked at in nearly 20 years (thought it was interesting at the time, in practice I've hardly done more than glance at it). It's not officially sanctioned, and IIRC, there were notices that these were not considered public APIs and could change at any time without warning. It was from the Win2000 era, so who knows how relevant it is anymore.

Regarding the reverse engineering, ReactOS may be in a legal grey area. If they're disassembling, there might be some legal issues, bit I dont know, I'm not a lawyer and dont follow Reactor development closely (just checking progress from time to time when mentioned here or other places).


Was it "Windows NT/2000 Native API Reference" by Gary Nebbett? (https://amzn.to/2XHhSFF)

That's considered a bit of a bible for NT internals in security circles.


ReactOS only officially allows clean-room reverse engineering. Sadly I don't know the best canonical documentation for this, so here's a blog post from 2006.

https://www.reactos.org/reset-reboot-restart-legal-issues-an...


Sadly that link is currently erroring out for me with a database error.

There is also a difference between stated policy and what actually ends up happening (not that I'm making any accusations, because I'm not). But what happens publicly isn't necessarily representative of what happens privately.

I mean, I can't contribute to it, because I have seen some source or disassembled components of Windows over the years. But excepting my public declaration of that here and now, who is to know?


Oh definitely, I choose not to speculate on that because again, its futile; it’s not really realistically possible to prove that it wasn’t done illegitimately. Clean-room reverse engineering can very well end up with indistinguishable results from just stealing code carefully.


Well said. I don't know if ReactOS has stolen code in it, but the Quora has some very shallow accusations. Besides, there are many ways of reverse engineering. Didn't Compaq famously have one team writing documentation for another team implementing? With the doc team scavenging info wherever they could.


Is Stardock still a thing?


Comments like these are eerily similar to what Microsoft had been saying for years about Linux - at best they had been infringing on patents but I seem to also recall accusations of the kernel containing proprietary Windows source code. This is just one individuals opinion but I would be really interested in seeing them provide more specifics - especially given the amount of MS source that has leaked/been open sourced over the years.


I don't recall accusations of Linux using Windows code, but these accusations (especially the "one has only to eyeball the code to see it was stolen") are reminiscent of the SCO troll-case, which was (indirectly but very likely intentionally) financially backed by Microsoft.

Without specific proof demonstrating that there is no way one could have come to write that code from publicly available information they are just FUD.


Upvoted. The history of Microsoft is well-documented, including the accusation of "stealing" code. Not sure why your comment has been downvoted.


Exactly. The author doesn't give any references, just says "Trust me, it is the same code". Maybe the author doesn't know that there are many resources with reverse-engineered descriptions of undocumented kernel structures and functions, for example [1] [2]

[1] https://www.nirsoft.net/kernel_struct/vista/index.html

[2] http://msdn.mirt.net/


It looks like ms has some sort of inside sect training on that matter. It is so easy to hear blames from ms staff on minor legal troubles that it just feels to be true. For one, I once read the thread on local ms support site that yelled at and almost banned a user for a story of upgrading to w10 and then downgrading back to 7. The problem was that his LICENSE was converted into 10-th one and now he LOST RIGHT to use 7. While maybe formally legal, for me this behavior is completely out of sane bounds.


Fascinating thread. You don't often see this kind of thing discussed openly.

One personal experience: years ago I worked on a project where I had the task (I will not go into why..) of re-implementing some tricky, non-documented functionality for which I had the source code for another implementation available to study, but also the goal to make it hard to prove from inspection of the code I wrote that it had been inspired by said original code. So in essence, I had the task we postulate the ReactOS developers had, if they indeed were using leaked ntoskrnl source as a reference. Ok, so when I had that task I did things like re-order the declaration of variables; change variable names; think of ways to refactor such that logic was split over a different number of functions; and on and on. Basically: I did a bunch of things designed to NOT permit the kind of "ah ha! : these things are the same" analysis that we see here. You might call this "derivation obfuscation".

So I'm a bit confused: either the ah ha analysis is flawed; or the RactOS devs just didn't care about being found out, right? Because if they had wanted to, it wouldn't have been hard to obscure, at least to some degree, that they were deriving their code from MS's original source.


When you've read and understood some piece of code, you're more prone to replicate it, even without realizing it. And ReactOS developers have obviously some extensive experience with Windows API, and it should be not suprise that any attempt to rewrite Windows will fall into the same trap.

I'm not saying that their code is "clean" or is a "rip-off". Obviously, I don't have a clue about it. Maybe they were just lazy and they did not care.


The basics are available in the windows driver development kit.

Many internal symbol and function names are available with debug symbols that have been released by Microsoft over the years.

Lots of undocumented internals you can learn about from this book: https://amzn.to/2XHhSFF

If the book doesn't match what you see (which isn't that often), then you just single-step and see what's happening.


Someone should get Alex Ionescu's take. He both used to be _the_ kernel guy of ReactOS, and he some wrote chapters in Windows Internals, some of Microsoft's deepest official public documentation.


Yeah, it seems like the guy answering here has no idea what he's talking about. Alex has a blog where he basically says that he'd spend the days writing Windows Internals and the nights reverse engineering the kernel.


So... doesn't that confirm what he's saying? How could you possibly "clean room reverse engineer" while you certainly are not in a clean room?


The guy in the article is talking about variable names from the source. You wouldn't get that from Alex's style of reversing.


It depends on what exactly the variables are referred to. FWIW, Windows DDK's example code has a consistent style as the kernel code (at least for windows 2000/xp). You won't have a thousand ways of naming variables once you get used to that style.


I assume that writing about Windows internals requires knowledge of said internals? Or do you think he never had access to the source code and has never seen anything, not even ideas of algorithms?


AFAIK, he never had privileged access while he was still commiting to ReactOS.


You can get knowledge of the internals and ideas of the algorithms from reverse engineering the released Windows binaries


Do you believe that is what Microsoft employees do while writing Windows Internals? I have serious trouble believing it. I'm not saying that it's not possible, just that it might not be the case there and these facts seem to confirm it.


Alex Ionescu wasn't a Microsoft employee, though.


I'm not so sure, maybe he was a contractor if that's what you mean (I don't see any real difference). He had a @microsoft.com e-mail address, which is a pretty clear indicator of being on the inside.


> Many internal data structures and internal functions, not exported anywhere and not part of the public symbols

I remember reading that even inside Microsoft, internal structures had to remain backwards-compatible because external, "very important" programs relied on them (violating the public API of course). I can't remember the source, unfortunately.

Now it would be interesting which aspects can affect compatibility and thus be part of a normal reverse engineering on the ReactOS side. For example, C usually does not remember structure names after compilation -- but "usually" is not always and depends on compilers, macros, and whatever somebody invented. The most interesting findings would of course be names that have absolutely no effect post-compilation AND were never published.

> If any of the presumed authors wants to chime in and explain the similitudes, I’m happy to change my mind, but be ready to answer some though questions about the origins your coding and naming styles, and all the design choices that you made and why you ended up architecting and writing things the way you say you did ;-)

On the internet, the burden of proof is on the accused.


"Many internal structures" that Axel believes are private and available only for MS employees, are actually published in the Internet, for example [1] [2]

[1] https://www.nirsoft.net/kernel_struct/vista/index.html

[2] http://msdn.mirt.net/


> On the internet, the burden of proof is on the accused.

True, unfortunately.



Hello from ReactOS team!

We don't use the leaked code from MS,

We don't use Windows Research Kernel,

We thank you for the PR campaign you have stared 'pro bono' for our project! ;D


OP wrote "If any of the presumed authors wants to chime in and explain the similitudes, I’m happy to change my mind".

Since you're from the team, could you provide some example explanations? Technical arguments are surely significant, rather than a purely rhetoric answer.


I don't think they need to. The OP blames ReactOS, the burden of proof is on him, if he thinks it's an issue. Otherwise it's called trolling.


Axel didn't provide any examples of allegedly copied code, so what can they explain?


This is an utterly insufficient answer to the allegation that you copied internal data structures entirely from the NT kernel, including (why?) their names.

Brushing this off as a free "PR campaign" only works if those NT programmers simply made everything up. That's possible, because we as the general public have no way of knowing. But it's hard to ignore that there are just two possible scenarios here, nothing in between.

Could you please clarify: If MS people say they do recognize verbatim code from NT in your source, are they lying? If they're not lying, what happened?


> This is an utterly insufficient answer to the allegation that you copied internal data structures entirely from the NT kernel

It's quite sufficient when you consider that, unless the allegation is made in a court of law, it's in ReactOS’s interest for it to be perceived as plausible. Being perceived as either an actual (if unauthorized) fork of Windows or as something that a Microsoft insider could reasonably mistake for one is not bad for ReactOS.

And if Microsoft is going to bring the allegation in court, no matter the merits, the lawyers defending ReactOS aren't going to want lots of detailed public statements to have been made first.

> Brushing this off as a free "PR campaign" only works if those NT programmers simply made everything up

It works and, I'd argue, is the only sane strategy so long as Microsoft isn't suing them. Because, among other reasons, without such a suit it is nothing more than a free PR campaign especially if it is true.

Admittedly, it doesn't maximize imaginary internet debate points, but while that maybe be OPs interest, it doesn't seem to be ReactOS’s. (Also, probably not Microsoft’s, who I imagine so long as they aren't suing ReactOS would prefer not drawing attention to it, either.)


I don't disagree with your assessment from a PR point of view, I guess I was just hoping the open source community would hold itself and others to higher standards. Being technically not provably guilty in a court of law is not exactly the peak of virtuousness.


Well, now please re-read your statement

Do you understand that internal data structures entirely from the NT kernel, including (why?) their names != verbatim code from NT in your source

The program code is logic first of all.

For the similarities of names and structures itself - the answer is in this already mentioned video https://www.youtube.com/watch?v=2D9ExVc0G10&t=9m53s from the 9:53 moment of time. TL:DW there are a lot of legal ways of obtaining names of variables, exported functions and structures


Never having had access to the NT kernel source, I don't actually know how deep these similarities go. When MS kernel engineers say those are not only superficial, I'm inclined to believe that until ReactOS have at least conducted an investigation.

It's a serious allegation that deserves more than a casual brush-off and some condescending statements about publicly available debug symbols. If MS is really full of shit, as I hope they are, they should be held accountable with more than a shrug and a smile. If ReactOS does contain copied code (and yes, that does include data structures as well as instructions), well then that should have consequences as well.


But how could ReactOS prove they didn't copy without having access to the NT source to show people differences? Unless there's some hard evidence, it shouldn't be up to the ReactOS team to provide evidence, but on the accuser.


Many "undocumented" Windows kernel structures are actually documented on third-party sites, for example [1] and [2]

[1] https://www.nirsoft.net/kernel_struct/vista/index.html

[2] http://msdn.mirt.net/

They were found, for example, in debugging symbols released by MS. And sadly Axel didn't provide a single example of code that allegedly was copied.


Heh, thats a very guilty looking answer.

True about the free campaign but that might backfire.


You didn't spell started correctly both on here and on quora aka thanks for the low effort canned response. Maybe try addressing their points directly. This is bad PR for you.


You mean the accusations for which the accuser refuses to provide any proof?


What "points"? Axel didn't provide a single example of what could be allegedly copied code.


Why would you call it a PR campaign if it wasn't favoring you? Just curious.


Not only is ReactOS getting a lot of coverage it otherwise would not have, but the core claim seems to be that ReactOS is actually such a good re-implementation of Windows that actual Windows engineers think it couldn't have been accomplished without using the Windows source code.


Probably because it makes far more people aware that ReactOS even exists.


because a lot of people probably just learned about ReactOS as a result of this controversy. "There's no such thing as bad press." -someone


The controversy is just unverifiable claims from a Microsoft employee. ReactOS comes out looking pretty good.


Even very dirty river water would still spin your mill, as long as the mill itself is good.

This guy literally confirms that one little open source OS could be as good as Windows $)


It's hard to know what specifically he means to have been copied. But it would not surprise me if a lot of "impossibly" identical structure member names have been published as ntoskrnl.pdb (kernel debug symbols).

I have also seen some pretty good reverse engineering of nt native APIs out there.


Alex Ionescu - Reversing Without Reversing https://www.youtube.com/watch?v=2D9ExVc0G10


The MS guy says there was no public NT internals doc. However many people refer to VAX/VMS Internals and Data Structures as being very similar. Both were architected by DNC.


Its bullshit too, there is not much public by MS but there is plenty of third party work done on documenting it.


>If any of the presumed authors wants to chime in and explain the similitudes, I’m happy to change my mind, but be ready to answer some though questions about the origins your coding and naming styles, and all the design choices that you made and why you ended up architecting and writing things the way you say you did ;-)

Can you taste the power-trip?


It would be interesting to see this engineer's statements tested in court. Microsoft are highly motivated to protect their source code so there would have to be a reason why they haven't. ReactOS is definitely NOT a mosquito to an elephant in violation terms if this engineers comments have validity. I'm thinking patents, copyrights etc.

The fact that Microsoft haven't acted for all this time is by itself evidence that could be interesting to present.

Contrast this with another chunk of law in trademarks where ignoring use and not challenging it strengthens someone elses laches defense.

(Usual disclaimer: I'm not a lawyer so these words are likely just words assembled by 400000 chimps for bananas)


I find the attitude more interesting.. on one side we have Linux embedded in Windows, more Linux VMs on Azure than Windows Server, MS employees proudly brandishing shirts with 'I heart Linux' and 'I heart OSS' and on the other side we have Windows Kernal developers saying their ideas were stolen.

Maybe it's a sign of the relationship between Azure and Windows teams?


I interned at Microsoft in late 2000's. The sales engineers had an "anti oss" budget. Many of these guys still work at Microsoft today, now "hearts" open source and linux...


Microsoft did try to hijack LinuxFest Northwest a few years ago, taking over half the event space, sponsoring the event and providing the afterparties (which were tightly controlled, compared to prior/following years where any LFNW attendee could just show up).

Multiple people I know stopped attending LFNW that year, and will not return. Hopefully other conferences like SeaGL don't fall for the Microsoft embrace scam, as it injured LFNW pretty badly.


They have no choice, Linux already won big time on the server, so the only way was to embrace it in a way that would bring them money. And they succeeded big time. Now most young developers don't remember all the dirty tactics used year after year by MS to kill all competition, whether commercial or open source.


Also consider the amount of people leaving and joining Microsoft since than as well as the possibility that lots of the old people a) didn't like everything Microsoft did back then and b) are capable of changing and developing over the years.


> Windows Kernal developers

Windows Kernel. To me best knowledge, Windows was never ported to Commodore 64 ;-)

* https://en.wikipedia.org/wiki/KERNAL


The comment is from one dude in late 2017 - it probably doesn't mean shit.

Like dude is talking about "there is absolutely no way on earth this was written from a clean sheet only from the available public documentation.", but that's not the bar for reverse engineering systems. It is legal (if you do it right anyways) to examine, use, and disassemble software in the process of reverse engineering.


What about EULA then? I know in some cases it is not enforceable, but I think it might apply in case of reverse engineering.


It has applied in the past: https://www.eff.org/cases/blizzard-v-bnetd


The Windows 10 EULA (XP has similar verbage) says:

c. Restrictions. The device manufacturer or installer and Microsoft reserve all rights (such as rights under intellectual property laws) not expressly granted in this agreement. For example, this license does not give you any right to, and you may not: (skipping a few) (vi) reverse engineer, decompile, or disassemble the software, or attempt to do so, except and only to the extent that the foregoing restriction is (a) permitted by applicable law; (b) permitted by licensing terms governing the use of open-source components that may be included with the software; or (c) required to debug changes to any libraries licensed under the GNU Lesser General Public License which are included with and linked to by the software; and

The EULA says you can reverse engineer to the extent that is legal in your jurisdiction.


If the EULA is a valid contract, and if you can prove someone involved accepted it before participating, you might have an action for breach of contract against that person, but if what they did wouldn't have been a violation of copyright on its own, the EULA won't change that.


This is the first sensible comment in here. A lot of jumping to conclusions.


Big companies are hardly ever coherent, let alone after a big pivot like Nadella’s.

The two things are also not exclusive: it might well be that MS “loves OSS” and that the ReactOS kernel was lifted wholesale from a Windows leak.

Besides, there are many who think projects like ReactOS or Mono are actually a net negative for the FOSS ecosystem, since they reduce the pressure on developers to switch away from Windows-based stacks.


I always heard that ReactOS had an ultra-strict vetting process, where no one who had ever so much as been within spitting distance of Redmond at some point in their life was allowed to touch the code, to support their claim of a clean-room implementation in case Microsoft ever decided to make a fuss about it. Seems like a lot of pointless effort if they actually did use leaks or university kernel releases. So either this guy is mistaken, my info about ReactOS is bad, or the project is run by people very ignorant of the law.


Why not consider the alternative that the process is just inherently flawed? (As in, it has flaws, even if its intent is good)

If I look at some leaked source code, then contribute to your project, you can't know the code is stolen unless you also look at it, which you can't do.

You can reasonably guess if a large chunk looks "unusual" but it can't ever be perfect.


I heard WINE has a rule like that but ReactOS does not.

But I also heard they borrow some code from WINE.


ReactOS had the same process.


I thought I remember reading on an FAQ a long time ago that they don't mind such contributions. Can't find it now. Could be wrong.

I myself am ex msft so that's why I was curious. I have not contributed to any of these though.

Edit: found it.

https://web.archive.org/web/20110707221707/http://www.reacto...

Section on tainted developers says: "Tainted Developers". There is a legal theory that is occasionally cited in the context of producing a work-alike implementation of a published copyrighted work. ... It is the position of the Project that this theory is invalid for a number of reasons. As a general rule, there is no reason that a developer who has seen non-free code cannot write logically similar code for ReactOS. Then talks about disclosure of NDAs.

I read that as: if you worked at MS, disclose your NDA, but it doesn't automatically forbid your contribution.


For MS's WSL, according to their devs, they only allowed themselves to read the documentation (external API). They didn't allow themselves to look at the code. Different case but also a form of reverse engineering.

It's the "internal data structures, macros" stuff that is problematic.


They looked at syscall surface primarily.


Honestly I always thought that that was an open secret of ReactOS. How else are you gonna achieve binary compatibility with a closed source OS. Of course there's symbols but those only get you so far.


In the past when porting netbsd to new architectures the approach was to first run a kernel with a binary compatibility layer to the existing operating system, be it something like OSF/1 or SunOS.

Not only was it doable, but people did it as a shortcut to getting a functional userland first.

It is easier on unix because they are similar to each other, but with enough persistence I believe Windows is possible too.

The steps are to get some kernel booting to a "native" userland, then add code to execute the simplest standalone Windows binary. Then when it makes a call to the kernel it will fail.

Generally tracing on Windows should give you a strong idea of what calls it makes to the kernel (I don't know, I have never developed on Windows, but probably it has something like ktrace/strace/etc.)

Once you can run a basic standalone programs, you try to add enough of a Windows userland to run bigger programs.

Once that's done, you can start to re-implement parts of the Windows userland. The symbols they export are public, you must implement all of them, and programs usually crash if you fail.

As for drivers, the Windows driver API is public, and other people have re-implemented it in the past. You can even pick up some of their work.


The information about many internal "undocumented" structures is actually documented on third-party sites. And if there are structures that are really not documented anywhere then you don't have to reimplement them because no developer is using them and they are not necessary for compatibility.


So you're using their "leaked" or reverse engineered information? That's just IP theft with extra steps.

If this is made to ensure compatibility with programs using those undocumented structures then it is a different thing. They become a part of API.

What goes around, comes around. Stealing code is on Microsoft records, as a simple "Microsoft admits stealing code" search shows. One can only wonder how much code the company stole that did not come to public knowledge.

https://www.computerworld.com/article/2522017/microsoft-admi...


Also many of the .wav files in Windows 98SE were edited with a pirated version of Sony's Soundforge, the metadata in the media files contained the Soundforge cracker's name.


As far as I could research, I only found articles about Windows XP. Close enough. https://www.tomshardware.co.uk/pirated-software-used-to-crea...


These .wav files might be made by person who works under Microsoft contract, not necessarily directly employed by Microsoft.


Which, of course, makes it perfectly acceptable?


No, but if Microsoft asked for wav files, and a contractor delivered them, is Microsoft at fault if the contractor used a pirated editing software?


Liable? I hope so. And then they can pass the buck on.


Why would Microsoft be liable?

Do you check if your butcher has illegally bought his knife? Should you be liable, if so?


Hmm... you may be right. But on the other hand, is the steak licensed and copyrighted?


It's not, but metaphors and analogies only go so far :)


That sort of shit is relevant only to lawyers. A salaried sound designer could have lied to Microsoft just as easily. The nature of the offense isn't actually being changed in any practical sense between the two scenarios.


I have always been confused as to what level of reverse engineering is legally acceptable in this case. Is having a sneak peak at one of the win32 dlls with IDA or Radare allowed? Or, are you expected to re-invent the wheel based on documentation and observed behaviour? In the case of the former, how can it be conclusively legally proven that it was not reverse engineered (from machine code)?


In the U.S. it is perfectly legal to reverse anything that you obtained legally.


Not if you agree to an EULA that explicitly prohibits it. See https://en.wikipedia.org/wiki/Bnetd#Blizzard_takedown_demand...

And https://en.wikipedia.org/wiki/Bowers_v._Baystate_Technologie....

Note that pretty much any EULA I've seen in the past 10 years has had an anti reverse engineering clause.


In that case, considering that even the undocumented methods are exported from the respective DLLs, these can easily be identified and their workings can be figured out by perusing the machine code. Access to source code, stolen or otherwise, is not really required. Also, as compatibility is ReactOS' goal, even identical bugs can't be credible evidence.


Reversing is one thing, distributing copies of reverse engineered IP is another. I am pretty sure that is not a good idea unless the terms under which you obtained the product explicitly allow it.


I appreciate the insider perspective (though I am definitely not qualified to assess these claims.) Unfortunately, it might be hard to prove these claims either way since it sounds like they have plenty of plausible deniability to fall back on.

What I will say is: I hope Axel's post doesn't negatively effect his career. We've all seen how controversial opinions and politically incorrect behavior have ended peoples employment in the past, and I honestly think its a ridiculous over-reaction.

I can't help but wonder if some of the answers here are driven by emotional, anti-MS sentiment. Some people seem way too invested, lmao. Just get some pop corn and see if anything interesting turns up. It might turn out to be a good history lesson.


Axel is a windows kernel developer, he has his pick of the job market. Microsoft alone is probably happy to cover him in dobloons just to stick around.


A rant that reads very much in the style of Open Letter to Hobbyists[1]. Typical MS.

[1]. https://en.wikipedia.org/wiki/Open_Letter_to_Hobbyists


Why would any paying customer use ReactOS over windows? There is no support (security patches automatically supplied ETC). Windows 10 is evergreen and new feature appear quite often. Microsoft loses exactly zero dollars a year because of ReactOS. People who don't want to pay for windows just pirate it. Those who do pay, get it buldled with a computer or laptop that they purchase, or are corporations who need support. ReactOS will always be a tinkerers hobby... Did they steal code? Did they reverse engineer so good that it looks like they stole code? Who cares really... Not Microsoft. Why would they?


> Why would any paying customer use ReactOS over windows? [...] Windows 10 is evergreen and new feature appear quite often.

Yes, exactly. Windows likes to update. Some of us want a simple system to run Windows programs/drivers that just works and doesn't demand you upgrade constantly to the latest version with all-new features that you don't have any use for. Granted, *nix+WINE is usually better for this, but there may be times when ReactOS is the better option.


> People who don't want to pay for windows just pirate it

Or they pay small fraction of the price on gray market: ebay.com, scdkey.com, allkeyshop.com, etc.


While I haven't given it a test drive yet, I'm quite excited about ReactOS. I grew up on Windows, but have been in a state of disappointment for the last decade while I watched Microsoft slowly mutilate or let rot so many of the things I like about it.

I hope by the time I have to give up Windows 7, ReactOS will have a viable alternative - or that it and the rest of the market will have forced MS to get its act together. Maybe give me the Windows 11 I really want.


> I hope by the time I have to give up Windows 7, ReactOS will have a viable alternative

Speaking as someone who has followed ReactOS from the very beginning - don't hold your breath. Their pace of development is absolutely glacial and very basic things still aren't working.


When I discovered ReactOS, I was passionate about it, I always downloaded the latest version available in order to test it in VirtualBox. But then, I noticed that the project was evolving so slowly, that it could never catch up his delay from Windows.


Why reverse engineer? The Windows code has been available via shared source license since nearly 20 years....


That license does not allow to repackage the system, rebrand it, or resell it, afaik.


I took a quick look at the structure of the WRK and ReactOS, as the MS developer suggested.

I can see what he means.

The first thing we observe is that the directory structure is basically identical. Yes, NT kernel subsystems are identified with short codes like "Ke" or "Ex". It's plausible that someone with a lot of NT knowledge would end up creating top level directories with these exact names, as is true of the WRK. But it seems kind of unlikely: the Wine sources do not show this pattern.

If we look at the Kernel Executive (Ke) subtree, we can see that there is a thing called a "Balance Set Manager". Both source trees define it in a file called balmgr.c - not only the location of the file but also the file name is identical.

https://github.com/Zer0Mem0ry/ntoskrnl/blob/master/Ke/balmgr... https://github.com/reactos/reactos/blob/master/ntoskrnl/ke/b...

It appears from the module description that the "balance set manager" is an optimisation of some sort related to reducing memory usage. Is this really something that needs a reimplementation with identical function prototypes?

Looking at the code of the identically named KeBalanceSetManager function, we can see that not only is the function prototype identical, but the order in which it does things is also identical. First it changes a thread priority, then it schedules a periodic timer callback.

Some of the local variables in these functions have identical names: PeriodTimer, DueTime, WaitObjects. Yes, these are obvious names. It's not a smoking gun. But it's not looking good.

Finally we discover that the ReactOS Balance Manager does .... nothing. It enters a loop which starts out by doing a wait for an event (fine, it's inherent to the task), and then switches on the result. But the code in the arms of the switch are commented out (the commented out code does a subset of the stuff in the NT code). The loop does nothing, just sits blocking in a loop forever. Why does this code in ReactOS exist if it does nothing?

It's the same story for the other big function in this file, KiScanReadyQueues. The code is virtually identical, line for line, with minor formatting and occasional trivial naming differences. Even the assertions are identical.

I'm not alleging anything specific or illegal, just comparing a small part of both codebases. However given what I've just seen, I wouldn't touch ReactOS with a barge pole. The Microsoft guy's complaint is entirely understandable.


Just wanted to comment that Googling "KeBalanceSetManager" gives this article [1] among others (the article mentions the leaked source code but it doesn't contain the code itself, just the description). Many internal function names were leaked with debugging symbols, and some people were reverse engineering those functions (for example, people who deal with malware).

The similarity in code can be explained by reverse engineering. For example, constant names in switch are different (TimerExpiration vs STATUS_WAIT_0).

> Why does this code in ReactOS exist if it does nothing?

Maybe because there is software that relies on the existence of this thread?

Regarding assertions and local variable names and same declaration oder, I don't know where they come from. That looks suspicious.

[1] https://github.com/bowlofstew/rootkit.com/blob/master/90210/...


> But it seems kind of unlikely: the Wine sources do not show this pattern.

Wine is not an NT kernel. Wine deals with the Win32 api, not the kernel.

> Looking at the code of the identically named KeBalanceSetManager function, we can see that not only is the function prototype identical, but the order in which it does things is also identical. First it changes a thread priority, then it schedules a periodic timer callback.

When trying to build an OS that is binary compatible with another, including the low-level like kernel drivers, surely it's expected that the prototypes be the same?

> The loop does nothing, just sits blocking in a loop forever. Why does this code in ReactOS exist if it does nothing?

Perhaps because the ReactOS kernel is note feature-complete, it's Alpha, so that part hasn't been implemented?


Microsoft has made great strides in opening up some of their code lately and the focus seems to be more and more on Online services. I hope one day, sooner or later, they go all in and open source Windows. That would be a great day.


Alex Ionescu on this subject: https://www.youtube.com/watch?v=ceQ6kT1ZMrk


Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: