Hacker News new | past | comments | ask | show | jobs | submit login
Most long-standing XHCI (USB 3.0+) issues resolved (haiku-os.org)
109 points by protomyth 14 days ago | hide | past | web | favorite | 58 comments



As of this writing, the XHCI spec is an 8MB PDF containing 604 pages. In contrast EHCI is 155 pages and OHCI and UHCI 160 and 47(!) pages, respectively. There's a lot of complexity in USB3, so it's not surprising it took this long to get the bugs out. The fact that "real" debugging requires an extremely expensive protocol analyser/oscilloscope also adds to that; without, you're basically restricted to "guess and test" and reverse-engineering other drivers to see the differences in behaviour.


(I'm the author of the blogpost.)

> without, you're basically restricted to "guess and test" and reverse-engineering other drivers to see the differences in behaviour.

Not entirely. If you look at the commits I linked in the post, you'll see that most of them fixed rather egregious spec violations. Arguably the spec could have been clearer; but still.

> The fact that "real" debugging requires an extremely expensive protocol analyser/oscilloscope also adds to that

USB packets can be sniffed in Wireshark. But for the kinds of bugs that this driver had, that isn't helpful; most of the problems were failure to initialize the controller or sending invalid command sequences. So that wouldn't be very helpful anyway.


> or sending invalid command sequences

Most of the protocol analyzers I've seen go out of their way to highlight invalid sequences and spec violations. IMO, that's most of their benefit.


No, I mean we were sending invalid command sequences to the XHCI controller, and as a result it wasn't sending anything to the ports at all.


There are probably ways of dumping PCI writes; I'm curious if any tracing of that sort was used (comparing traces to known working OSes), or if it was all about comparing code to the spec.

We've gone through similar rounds of debugging our XHCI bits on illumos. PCI and USB sniffing hardware is expensive and we have access to neither.

You can get a long way by becoming comfortable with the specification, reading and getting review on your source, and by using the debugging tools on your OS (we have MDB and DTrace, etc).


Using a VM is another way --- a virtual USB controller can log how it's being manipulated.

is there a hall of shame for these broken drivers, devices and vendors?

You should see the Bluetooth spec

4kilopages, before you even get to actual profiles.


To be fair, they've realized BT was all a huge unfixable clusterfuck and started anew in the later versions. I've heard that USB is worse though, especially AUSB-Audio, -PD and -Video.

    >  and started anew in the later versions.
entirely not true

source: been working on this for any many years :(


I've only heard that 2nd hand, so forgive me for spreading bullshit. The current situation is that we have Bluetooth Classic, high speed (used for AirDrop?) and BLE, right? I guess the confusion was caused by the "BT classic" terminology, which if I understand you correctly means is here to stay?

classic is here to stay because car kits LE uses a lot of the same HCI commands and some of the same protocols (l2cap).

airdrop uses LE to establish connection


That's not quite fair. They're not even printed on the same size page! XHCI has much bigger margins. Compared to EHCI, it's 4 times as many pages, but only 3 times as many words. Plus, adding diagrams can help clarify, rather than adding complexity. And XHCI now has a glossary!

Anyway, is that a lot for a technical specification? Most programming languages I've used in the past year have had longer specs than that (if they have one at all). Even plain C is over 500 pages now.

The biggest concern I have here is requiring an oscilloscope. Nothing here looks like it should require an actual physical layer. Why was USB designed to conflate these layers in the first place? The whole OSI stack might be silly but separating the physical layer has always been a great idea. When I upgrade from 10/100 ethernet to 10/100/1000 ethernet, I don't have to re-test my web server and mail server.


> The whole OSI stack might be silly but separating the physical layer has always been a great idea. When I upgrade from 10/100 ethernet to 10/100/1000 ethernet, I don't have to re-test my web server and mail server.

No, but you had to change all your network cards on both ends of the connection, and of course there were new drivers for the new chipsets. Same with USB.


Also, a lot of the XHCI hardware is itself buggy. Linux has so many workarounds for different issues with XHCI host controllers that they had to increase the flags variable indicating which needed to be enabled to 64 bits because they'd run out of bits: https://patchwork.kernel.org/patch/10421969/

Gentlepeople... the time to try Haiku is now! get an image, and boot it. Out of 10 machines with different ages, all but one get to desktop in a few seconds. For the one that gets to a blank screen equally fast, make sure to press <space> on boot and choose vesa.

Browsers? Usable for the focused mind, lots of wip... For me, I have text editors, compilers, a terminal, and aws cli... I can wish for more, but I am happy with this.


Is it safe to boot on an expensive laptop like a MacBook? I remember when Linux had missing drivers there were risks that you'd overheat your machine because the laptop manufacturer was controlling a system fan in some nonstandard way.


Macbooks >= ~2013 or so will run into this kernel panic shortly after boot: https://dev.haiku-os.org/ticket/13189

2012 and earlier Macbooks pretty much work, I'm told (though you will hit the remaining NEC/Renesas bug I mentioned in the blog post, so use a USB2 drive until we manage to debug that.)

But yes, generally our hardware support is not quite as good as Linux's, so you will have a mixed bag in Macbooks. High-end PC laptops (e.g. Dell XPS 13) are fine.

EDIT: According to the comment below, it seems I was misremembering, and these problems occur on 2015-and-up Macbooks, not 2013-and-up.


I run Haiku natively on MacBook Pro 2014 (11.3), on SSD with EFI boot. Have working audio (headphones only), wired ethernet (no WiFi), very usable Hi-DPI display (2880x1800) with basic framebuffer driver. Retina MBP up to 2015 should all work like mine. Good enough for developing apps.


MacBooks still control the fans in some nonstandard way so you need to install some software to control the fans for you, not sure if it exists for haiku though


I've booted Haiku on a 27" 5K Retina iMac, 6th gen i7 without too much hassle. The AMD graphics didn't work, so needed VESA mode.


I just tried it for a few weeks, ending last week. It wasn't bad at all, though individual apps had various bugs such that the experience often came down to same-app-on-Haiku vs. same-app-on-Linux. Also, while I liked being able to join different apps into one tabbed window, the other workspace features in Linux DE's seemed even more useful.

So: I was surprised at how easy it was to end the experiment early and try out another Linux distro, even though I'll probably try out other Haiku releases in the future. The boot time and single-user configuration was definitely a nice break.


> the other workspace features in Linux DE's seemed even more useful.

Such as? Haiku has the "workspaces" concept too (https://www.haiku-os.org/docs/userguide/en/workspaces.html), though perhaps not as full-featured as Linux's.


Yes, I think it's the full-featured aspect that made a lot of difference in this case. Though I didn't have any _workspace_ problems in Haiku, it was more like the panel concept in e.g. XFCE is simply more refined.

A really basic difference example though was that the default window move/resize RMB shortcut modifier is simply Alt in my Linux install, as opposed to Ctrl+Alt on Haiku. On a cramped keyboard this combination was fiddly. I also swap caps and CTRL on this keyboard in Linux and I hadn't checked yet if this could be accomplished. There were lots of things to look into.

I appreciate your response, and will probably try out Haiku again in the future. I liked that it was easy to install and test. Also as someone who uses QMMP a lot, I was very pleasantly surprised at the neat Be-style skin that came in its Haiku install. :-)


> I also swap caps and CTRL on this keyboard in Linux and I hadn't checked yet if this could be accomplished. There were lots of things to look into.

This you can already do; see Keymap preferences: https://www.haiku-os.org/docs/userguide/en/preferences/keyma... -- you should be able to just drag & drop the keys around to swap them. Or you can edit the keymap file directly to completely customize it.

> the default window move/resize RMB shortcut modifier is simply Alt in my Linux install, as opposed to Ctrl+Alt on Haiku. On a cramped keyboard this combination was fiddly.

We have a ticket somewhere to revamp the hardcoded shortcuts into the "Shortcuts" preferences pane; but nobody's done this yet. Eventually...


Haiku is an amazing project. I just wished they had like a shop to purchase t-shirts, stickers, plushies and whatnot like the FSF. I'd buy in an instant, and it probably would be great for their donation goals.



That's awesome! But I wish it was more visible in the website, I couldn't find it at first glance


I just got a sticker, but I think you guys could improve on the work, I see potential with the logo and variants, and a mascot would be very cool. Just saying, well done merchandising sells well. Good luck!


So I guess Haiku development isn't going that well if they're struggling to get USB 3 sorted.

Shame, it seemed a promising project.

Edit: Okay my assessment was based on this, the fact that there only seems to be a beta release for download, and there didn't seem to be a decent web browser at some point fairly recently ( there was a story on HN?). Are these misunderstandings? Do we have different definitions of 'doing well'?


To compare, Linux kernel had recently a significant problem by which a lot of the users of Ubuntu 18.04 LTS were affected. It was related to the use of the DPMS signaling, standard for which exists since 2003, if I understand correctly:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1813663

Programming for real hardware is simply harder than the most can imagine.


Responding to your edit: I downvoted you because your comment was not constructive and basically was just chastising a project composed entirely of a few geographically-separated volunteers.

It's very honestly amazing what they've accomplished.


Wasn't intended to chastise them at all.

Yes it is amazing what they've accomplished.

I was mourning the fact that despite all the mind share, all the talented developers, they didn't manage to produce an OS that just works (tm). I'm cognizant of the fact Linus Torvalds started out very in similar circumstances. I don't know if it is Linux taking too much air out of the room, or if in 2019, building the OS that just works (tm) is just too bigger job for volunteers. That is the shame.


> I was mourning the fact that despite all the mind share, all the talented developers, they didn't manage to produce an OS that just works (tm).

We have ~10-15 major active contributors, and with that, we can basically compete with "Desktop Linux" in terms of functionality. Not everything works or is polished to the degree we'd like ... but it's nearly there.

So then who is the one wasting the "mind share" -- the Linux DE space, with its hundreds of developers spread across dozens of projects, or us, who can do basically the same thing with a fraction of the time and resources?


"basically compete with "Desktop Linux" in terms of functionality" On what terms?

"Not everything works or is polished to the degree we'd like ... but it's nearly there" What does nearly mean?

Please don't interpret those as being argumentative rhetorical questions, I take the we to mean you're a contributor, so you have the right to define those.

The first Alpha was in 2009? So personally I'd expect you to be there by now. And I'd expect an OS that just works, which would include working USB 3, and a decent web browser.

So when is a stable release due, (I would like to try it)?

What are your aims for the OS? Do you want an OS that I can recommend grandma puts on her computer knowing it will work, or is it an OS where I'll have fun diving into config files? Reading the website etc I've understood the longterm aim to be the former. If I'm wrong please let me know, so I can calibrate my expectations.

Thanks for your reply btw. I do want you to succeed.


> "basically compete with "Desktop Linux" in terms of functionality" On what terms?

Err ... I don't know what you mean?

> "Not everything works or is polished to the degree we'd like ... but it's nearly there" What does nearly mean?

Uh, it means there are still a lot of bugs, and some missing major/minor features in certain areas (e.g. GPU acceleration is a massive TODO / WIP; I have some work on it out-of-tree but it's not in yet.)

> The first Alpha was in 2009? So personally I'd expect you to be there by now.

Uh, excuse me? By whose standards? We are, as I said, 10-15 (well, then it was a little more) people who do this for fun. We have lives outside of open source development, you know. We aren't paid to do this.

> And I'd expect an OS that just works, which would include working USB 3, and a decent web browser.

So you expect a product. Great. Who's paying?

> So when is a stable release due, (I would like to try it)?

What's stopping you from trying it right now...?

> Do you want an OS that I can recommend grandma puts on her computer knowing it will work, or is it an OS where I'll have fun diving into config files? Reading the website etc I've understood the longterm aim to be the former.

That is correct.


Ok, so the long term aim is an OS I can put on Grandmas computer?

Ok, then I don't think it's fair to start being hostile to people when they expect USB 3 support and a web browser. I don't expect a product, that's what you're saying you're offering.

You said you're nearly there, I think it's reasonable to ask when that might be. I'm not asking for a cast iron guarantee to the day, but 1 year? 5 years? The thing stopping me from trying it is lack of stable release.

" "basically compete with "Desktop Linux" in terms of functionality" On what terms?"

I mean do you want to be judged as a hobby OS, or something to stand toe to toe with Linux on every measure. Theres no point me complaining USB 3 doesn't work if your only aim for the project is something to do on an evening. You've indicated that you want it to be something to put on Grandmas computer. Fine, but that sets a much higher bar, and you can't say "fiddle with config file" when something goes wrong.

Ps I'm still detecting hostility, I hope its because you vehemently believe I'm wrong and you can succeed (I hope I'm wrong, and you do succeed)


Self-entitlement, thy name is benj111 apparently. Nobody needs to measure themselves against your lofty standards for them; they know what they're doing.

If you want them to succeed, help them, get involved, don't sit on the sidelines jeering at them for ten to fifteen people not being at the same level as thousands of contributors.


Why am I self entitled? I'm not measuring them against my lofty standards, I'm measuring them against their standards.

If they want to put it on github, with an announcement saying they've done a thing for fun, what do you think? I'll judge it on those terms. If they say they're aiming to create an OS that I can put on my Grandma's computer (which they have, read the thread), then its only fair on them to judge it in those terms.

I've been complementary and wished them well several times in this thread, because yes there are only 15 of them, but that doesn't change the fact, that they haven't yet built what they said they were going to build. That isn't jeering, and it isn't patting them on the head and telling them that it's the taking part matters, either.


A modern os is vastly more complex than what Linus was working with in the early 1990s. Recall that the web had just been invented at that point.


I’m not sure I buy this.

X86 hardware diversity exploded shortly after Linux took off, which gave more popular operating systems a huge incumbent advantage.

However, now that Moore’s law has petered out, you can target Intel’s current mainstream SoC, and you have a bootable product.

Hopefully this translates into more competition in the operating system kernel space.


I'm not sure I see the link between your comment and the one to which you're replying.

GP is saying that the expected components that make up a modern operating system, including things that might be seen to be 'trivial' such as full USB 3.0+ support; a full, robust networking stack; the sorts of security and protections against modern methods of attack from bad actors; all on top of modern, efficient, and possibly even forward-thinking APIs that make developing for the OS efficient and even a joy for developers — these things are neither trivial, easy, or quick to do.

Whereas you're talking about just hardware, which is not even close to the whole story when it comes to an OS in 2019.


For the record, I didn't say 'trivial'.

I said basic, in the sense of USB 3 is a basic component I would expect to see in an modern OS. I did not, and do not claim that it would be trivial to implement.


You're not even the person about whose comment I was referring.

It was either me or Jnbiche, and jnbiche is saying the opposite of what you're saying he's saying, whereas if you squint a bit I did, and I may not have been your gp, but was the gp of the person you were referring to. So you see my confusion.

I've been using haiku as my main work os for a couple of months now. Granted, there are issues, but none of them too annoying to prevent me from getting things done... except for xhci.

It would seem like this specification has been a pain point for many other projects before. For a os with only a few contributors doing work on their freetime, I think it is going great. (My opinion, like mouths... everybody has one)


USB 3 support seems.. Basic at this point? The post mentions 2012 when chips became prevalent so this isn't new. I also seem to recall an issue with getting a decent web browser recently?

"For a os with only a few contributors doing work on their freetime, I think it is going great "

Oh definitely, I bear no ill will, and would like the project to succeed. But can you make a full featured desktop OS in your free time. There's another comment on this thread pointing out the spec is 600 pages long. I can understand how if you're doing it as a hobby that would take time.

If someone built a car in their garage I'd be impressed. It may have taken 10 years, look ugly and leak oil but its still an achievement. I wouldn't buy it though, I don't want a car that leaks oil, and doesn't have crumple zones etc.

I guess what I'm (slowly) stumbling towards asking is, is this a hobby OS or a 'big professional' OS? Reading between the lines you seem to be happy with a hobby OS, if so great, but that isn't what I want, I'm not sure that is what it was hyped to be.


> USB 3 support seems.. Basic at this point?

I think you underestimate the sophisticated complex problems of USB 3. Even for a hobby OS this is a massive achievement.


It is a massive achievement just like the home made car I mentioned.

The car has to compete against Tesla, Ford, Toyota though, does it cut it? Does Haiku cut it against Linux, Windows, the BSDs?

Maybe I have unreasonable expectations that it should.

The car can exist for its own sake. The builder had fun making it after all. I thought the idea was that we were all going to be driving round in our own home made cars.

So am I judging it on the engineering skill of the builder, as his hobby project? Or an I judging it as my next potential car?

I can forgive that home built car not having crumple zones, radio, opening windows. They are hard to do, as a car buyer they are basic things I expect.


If your "assessment" is based on the title of a blog post and some opinions you have about their download page... do we really need your assessment? It's a volunteer project. If you want it to do better, help. If you don't or don't care, why comment on it? Who are you helping? What is your commentary improving?


Lots of commentary doesn't improve anything.

What has all the discussion on HN today improved or helped?

If you don't wish to be reading other peoples assessments then you've come to the wrong place.

If you think I'm wrong, please correct me. I've put my opinion and reasons why.


I have Haiku as sole OS on a Thinkpad, not my primary machine but some days I'll work the whole day on it no problem - it's a really enjoyable experience in fact.


Mr. Page, it's good to hear you like our work! Perhaps you could lend us some assistance in our world-domination goals...? ;)


I haven't tried, and I won't at least for a few months. But I looked at the list of installed apps, and I couldn't find a calendaring (iCal) app. If that exists, I and many others I think, could use Haiku "for real". Realistically, I only need a terminal and SSH (check) a web browser (check), email (check) and something with iCal. Since many organizing solutions (such as Todoist - no affilition, I just use it) has iCal integration, iCal gives you a way to integrate and sync with your phone, be it iOS or Android.

So if I had all that, I could use Haiku as my home/hobby tinkering machine. I would still need a Mac, Linux or Windows machine to do some other heavy lifting, but I already have three computers at home, so no problem.


There's a relatively sophisticated Calendar app already, indeed: https://github.com/HaikuArchives/Calendar/

It had working Google Calendar sync at one point, dunno if it still is. It could definitely use some work, though.


I suspect their work optimizes around a relatively narrow band of supported hardware.

The Beta 1 release, when I tried it on my Ryzen-based desktop, wouldn't get past the splash screen, stalling out at things that seemed to revolve around the SATA controller.

Meanwhile, pop the same flash drive in a Thinkpad X230t, and it booted and installed fine, and even Wi-Fi worked without a significant hitch.


> The Beta 1 release, when I tried it on my Ryzen-based desktop, wouldn't get past the splash screen, stalling out at things that seemed to revolve around the SATA controller.

Did any icons light up? There was (is) an undiagnosed issue that is preventing BIOS booting from working on Ryzen. I know one of the other devs runs Haiku on his Ryzen via EFI and says it works like a charm.

> I suspect their work optimizes around a relatively narrow band of supported hardware.

Not by design, but just by circumstance -- we developers optimize for the hardware we have and use, and not many others tend to submit patches, so that's the way it goes.




Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: