I imagine also how more difficult it would be for someone to reverse engineer the drivers for something a lot more complex than a VGA-to-webcam box.
Just look how locked down Broadcom documentation and even datasheets are. Everything is under NDA.
HW business make money by selling more HW so making you buy their latest product just for the latest updates even though the 10 year old one still does the job is a viable business model for them.
Source: I worked in the hw industry and it's how they think.
As for datasheets, I feel like from certain vendors, they are "living documents" and every customer gets a different datasheet. You might mention in a meeting with the sales engineers "oh, we need a 100MHz SPI bus" and then they will edit the datasheet to say that the SPI bus works up to 100MHz. This is why there is an NDA, every customer gets a different datasheet exactly tailored to their needs.
>I believe this is the reasoning NVidia gave for not open-sourcing their Linux drivers
Do you happen to have a source for that? Or at least an explanation of why you think this is the case?
Even if the performance isn't the best, the openness is worth that price to me. I don't feel like the stability of the rest of my system is hampered by a third party that doesn't seem to care.
The UX of the "fglrx" drivers tended to be pretty awful, all told; typically worse than nvidia's proprietary drivers, so they were wildly unpopular among users. The performance gap eventually started closing, to the extent that some games/apps would be faster on the "radeon" drivers, and some on "fglrx".
Eventually, AMD switched to the AMDGPU model, where the kernel driver is always FOSS (and part of the upstream kernel), and there is the FOSS userland driver as part of Mesa for consumers, and a proprietary certified OpenGL driver for workstation users.
(I believe Valve also had a hand in this somewhere along the way - they were interested in shipping Steam Machines/SteamOS and understandably were keen on having a stable and fast graphics stack and so contributed to Mesa.)
>The UX of the "fglrx" drivers tended to be pretty awful, all told; typically worse than nvidia's proprietary drivers, so they were wildly unpopular among users.
This is my memory of the time. Both options being undesirable and proprietary but nvidia's at least working better.
"without looking at their financials" does not make sense. Do they open anything in the Windows space? If a HW company wants to sell their stuff to the Linux desktop users they are generally forced to open the source, not because of good will. In terms of openness Intel does it a far away more. nVidia seems to be only exception.
Freesync comes to mind and I think they had something related to physics/hair rendering.
There is no requirement to open source on linux.
Nvidia does not. It just means you have to constantly update your driver for the latest kernel version and spend time keeping up to date with all of the latest developments like wayland.
This is wrong. If you publish a Linux kernel module, you might be obligated to release the source. Fundamentally, this is how Linux differs from *BSD: Linux is intended to be generally hostile to binary blobs.
But it's not universal. Linus thinks nVidia's module may be exempt from the GPL's 'virality' because it wasn't originally written specifically for the Linux kernel.
See also https://stackoverflow.com/a/737030/
I mean, I wouldn't force the manufacturers to keep supporting the hardware after a decade, but shouldn't there be a law compelling them to open source the drivers and firmware upon end-of-life? If a product reaches end-of-life, surely the manufacturer already has a demonstrably better, upgraded version of the product to attract new sales?
We're probably going to see more of this in the coming years around SecureBoot/TrustZone on ARM.
Would be cool if manufacturers would make old versions of sourcecode public after a couple of years, and the final version when a product goes out of production.
But in the hardware game there is a lot of counterfeiting. There is rarely any shame or consequence when it comes to outright IP theft. Making it easier to access the firmware makes it easier for someone to counterfeit your widget.
You think the public facing representatives (management etc.) are morons, because
you think their strategy for working in the same space sucks unless
you think your company is totally blowing it and you are on your way out
but you don't think their programmers are morons so much (except when you examine site and it performs worse than yours for obvious reasons that you fixed on yours) but
if they roll out a really cool feature you can implement in your product you will write code to implement that feature and not wish you had their code because anyway your stuff is incompatible codebases unless
you get bought by them or they get bought by you or you both merge because you are in fact compatible and this way something beautiful will emerge
This certainly caused FTDI a lot of trouble.
> when a product goes out of production.
By the time this happens, the company has lost their throwaway source code.
Interestingly, all the Saleae clones seem to be based on the Cypress USB2 devkit.
I also wouldn't even be considering buying a real Saleae device without having used a $5 clone first.
Of course they may not want any of this since it means they can sell a new phone every 2 years.
ReactOS is for hobbists that don't mind if their toy OS crashes every 3 minutes or so.
I own a Fujitsu Scansnap s1500. It's out of support, and the existing drivers are incompatible with Catalina. I now must pay a 3rd party $100 for drivers or fiddle around with a Linux scanner server or similar. Never again will I pay $400 for a Fujitsu scanner, that's for sure.
Even if it would make financial sense long term, how likely is it that the person making the decision will be there in 20 years to deal with the mess?
On the other hand, it will look much better on their next quarter results.
PC and peripheral manufacturers in the embedded space often provide availability guarantees e.g., "this version of this hardware will be available for sale at least until YYYY."
Sometimes it matters, usually it doesn't. But it does at least make ordering spare parts easier.
It's more that buying old PCs is safer than trying to reverse engineer drivers. Buying enough extra hardware to have spare parts for the life of the product is a pretty well known and cheap way to insure no down time.
I have experienced even when the original driver writer (me) is still on staff, it can be less expensive to just keep buying old-spec PCs than rewrite the code for a new OS. I make the company more money writing new code than supporting code that's a decade old.
On a side note, I wish there was a more standard way to deal with Linux kernel modules. The ABI incompatibility leaves a ton of dead old hardware out there for non-maintained kernels. In an ideal world without tons of binary blobs and .o files out there; this wouldn't matter. You could recompile everything for the latest version.
You might have to fill in some missing symbols or swap out deprecated functions, but it'd be considerably easier than this. That was the ideal behind open source; that we could always be able to figure out how our stuff works, and it seems like today we only have open source middle ware, and so much close source stuff build on top of open source stuff.
(Edit) It doesn't appear that he decoded the bitstream format though, so some more work would be needed before you could use it as a general-purpose FPGA.
But while the article says he got a lot of them (at least 3 from the photo) for $25 total, it looks like they're more typically ~$50+S/H each on ebay, which is less exciting.
The same company seems to have moved on to 'av.io'-branded USB3 devices now, which from a quick glance/binwalk of the downloadable firmware package (inspired by this article) appears to have FX3 USB Super-Speed controllers, and Spartan FPGAs. But it's designed to store the FX3 firmware and FPGA bitstream on the device itself (to allow it to boot up as a UVC webcam with no drivers required on the host, like this article was hoping would be the case for the older device), so it would require a bit more reverse engineering to figure out how to kick the FX3 into firmware update mode to replace the FX3 firmware with a custom bitstream uploader. But it's a ton more expensive, and who knows what generation and size of FPGA or how much and what sort of memory it would have.
The last one alone, CY7C68013(A) has always been an expensive chip (like $10 a piece if you just want a few).
What is funny is that I wasn't aware it existed in QFP-56 package, it is not supposed to exist :-)
Worst package I've had the pleasure of hand mounting.
It also provides superior electrical properties like low-inductance.
The bitstream format isn't anything special. It's whatever Xilinx says it is for the Spartan 6. Any bitfile generated by ISE for the XC6SLX16 could be uploaded via the (generic) Cypress FX2LP chip.
This is all relatively old hardware, very well-documented and well-understood but still quite capable. It has a lot in common with the earlier-generation Ettus USRP receivers that used the FX2.
A lot more, actually. Manufacturer's specs for the ISL98002CRZ-170  say it'll do 170 MSPS, with an analog bandwidth of 780 MHz. It's actually a rather impressive ADC! The question would be whether it can be convinced to sample continuously.
I wasn't aware of that. This is even better of course!
Sadly it seems that was just a stroke of luck, since I can't find anyone else selling them anywhere near that cheap now
That diagonally skewed image is familiar to me from old game writing days when sprites were written straight to screen memory - you soon learn that if it comes out skewed, you've probably got an off-by-one bug on the horizontals.
Nothing as impressive or interesting as TFA, just enough to let the thing boot on Linux to the point where proprietary half-working drivers can take over. It'd be a stretch to call my epson scanner abandoned hardware, but considering the quality of official drivers, only a small one.
While it's still based on the 2.6 kernel, it's probably good enough for the basics. Not sure there is a more modern source.
(Robert Love's Linux Kernel Programming was also a nice introduction to how the Kernel works, though it might be outdated in some areas now as well)
LDD3 goes way more in-depth and teaches you about the specifics of many different classes of device drivers, their interfaces and data structures, as well as where to find the appropriate in-tree documentation.
(both LDD3 and LKMPG are written for the 2.6 kernel. it's a shame there are no newer resources of the same caliber.)
It's part of tldp (also google searchable)
I wonder how someone from the outside is supposed to know that's the magic string to google.
The first result for the utterly mundane, plain English "how to program linux drivers" brings me to this tutorial: https://www.apriorit.com/dev-blog/195-simple-driver-for-linu...
Linked from that tutorial is a hypertext version of the content in that PDF: http://tldp.org/LDP/lkmpg/2.6/html/lkmpg.html
That's a beautiful way to put it. You've distilled into one sentence a skill that I've been asked about many, many times (as the token IT guy).
I tried "how to write device drivers under Linux" (in quotes) which showed one result and I ended up at this exact post.
The same way people learn to do many other similar things.
Now I don't know how to write userspace USB drivers, but there has to be something between 0 knowledge and just "trying and immitating"[SIC] kernel source code. At least I'd be happy for more pointers on similar resources as the submitted link, but more beginner oriented.
For learning more about how USB itself works, I highly recommend Jan Axelson's USB Complete. I used an older version of this book to build a from-scratch USB device (on an STM32F0 but not using the built-in USB example code) and libusb-based usermode driver for it.
And what to learn depends on what driver you want to write, so you best start by looking into the code in the relevant subsystem, and read it, understand it, and use it, and learn what you need on the go, based on the needs you discover you have for your driver. Then you may realize what you need to learn, and if you can't grok it from the code, go for the books/presentations/linux kernel docs.
So diving into kernel code is a way to discover what you'll need to learn to write your particular kernel driver. Kernel's api surface is huge, and you'll never learn/need anything, and it's best to only learn what you'll be using soon. And this is one way to discover what that might be for your driver.
For kernel wide resources that the kernel offers (like threads, workqueues, locking primitives, of interfaces, kobj/device interfaces, sysfs attributes, timers, etc.) you can learn them from books, but it's also just as effective to learn them by example simply by going to https://elixir.bootlin.com/linux/latest/source and searching for all references of given function, and learn by example/implementation.
USB userspace drivers are not kernel drivers, so there the situation is different. Interface for writing them is part of kernel-userspace ABI, so you'll be able to use them even with half a decade old guides on the internet.
I aggree that to learn C, you'd probably not start with reading Linux kernel code.
That's not the correct analogy, though. Learning to write a kernel driver would be more analogous to learning how to extend Kubernetes.
So if that was the goal, and it would be known that Kubernetes internals and its interfaces to extensions are constantly evolving/changing, that there'd more value in diving into Kubernetes code, instead of reading up on possibly outdated howto guides or whatever.
There are books such as "Linux Device Drivers", which are great resources, and there is a lot of existing code out there to read too.
The person who asks "How do I do x?" without having appeared to carry out the minimum of self-searching themselves is often impossible to help in any detail anyway.
First of all, I had to realize that the keyboard was in fact controlled via USB. It seems obvious but other Clevo laptops did it via ACPI:
I assumed my laptop worked the same way and wasted a lot of time dumping my laptop's ACPI tables and trying to figure them out. I even installed some Windows tools to decompile the WMI buffers. Didn't find anything. I was about to give up when someone on IRC set me on the right path and I was able to make progress. Even though I saw my keyboard on lsusb output, I never thought to look for a USB protocol. Learned this one the hard way.
Before I started, I emailed Clevo and asked for technical documentation on the keyboard. Their marketing department replied: "Ubuntu was not supported". I emailed Tuxedo Computers (mentioned above) and they were much more helpful: their developers shared a fan control application they wrote with me! It was great but the keyboard drivers were missing.
So I decided to reverse engineer it.
While not as complex as the VGA-to-USB device, the process of reverse engineering the keyboard was similar:
1. Boot Windows 10 with the proprietary driver
2. Start Wireshark
3. Capture USB traffic
4. Use the manufacturer's keyboard control program
5. Correlate the data sent to the keyboard with what I did
6. Document the structure of the messages
01 = some kind of operation code
kk = target LED
rr = red
gg = green
bb = blue
cc, 7f always surround all bytes (part of USB protocol?)
This user space driver gave me access to most of my keyboard's functionality on Linux. I used to have to boot into Windows in order to configure the keyboard lights -- not anymore!
The keyboard also generates some custom key codes that are used by the proprietary Windows driver to implement some Fn hotkeys. I know this because the kernel logs these unknown key events. To implement this, I'll have to make a real kernel module that maps them into something user space can make use of.
To this end, I've been studying the Linux kernel's documentation:
The input_mapping, input_mapped, event and raw_event hooks seem especially relevant to my case: the custom driver just needs to process the custom key codes and let the generic Linux keyboard driver handle the rest. Might as well add sysfs entries for the LEDs but I don't know how to do that yet.
I expect to end up with something like the open razer driver:
I used this site to gain a better understanding of USB itself:
I'm not asking this passive-aggressively, if someone knows the answer to this I'd really like to hear it.
And at least for GPUs, drivers are pitched as a feature and may contain secrets worth protecting in the eyes of the business.
In some environments, if the code was not developed using a particular process (my experience is with regulated medical devices), it may as well not exist, because putting it into production would be in violation of Federal law.
There are also small niche products where most of the IP is in the driver: lose control of that and you're on your way to being out of business.
What are they, out of curiosity?
Looking through his archive, I recognize his bgp battleship post from last year on HN.
It's disappointing to click on the comments for a post and no one (especially the submitter) has told me what I'll learn/experience. And yet, there it was, four hours old, ten points up and no comments. So I give it a skim and it's really good. Now I want others to read it. Spending five or so minutes polishing three sentences is basically free for me and positive sum (unless I do a bad job).
Cynically, tl;dr submission summaries are how I've chosen to feed the beast until it grants me downvoting power.
That's pretty much n-gate summary format :-)
Anyone know why?
There was apparently an HN thread as well
This document shows how unfriendly is Linux to third-party code. Either you contribute into giant monolitic kernel, or your device won't work with next major kernel release.
> Depending on the version of the C compiler you use, different kernel
> data structures will contain different alignment of structures
That doesn't make sense. How does Microsoft use structures in their API then?
Furthermore, unlike Windows software, you cannot compile Linux kernel with any compiler except for GCC . It doesn't comply with C standards.
> Depending on what kernel build options you select, a wide range of
> different things can be assumed by the kernel
> - different structures can contain different fields
That looks like poor design. Why care about non-standard configuration? Let those who use it solve the problems themselves which would be the true open source way.
> Linux runs on a wide range of different processor architectures.
> There is no way that binary drivers from one architecture will run
> on another architecture properly.
Supporting i386 is enough. Nobody is going to insert a PCI device into a smartphone. If there will be new CPU architectures, one can use i386 emulator.
> As a specific examples of this, the in-kernel USB interfaces have
> undergone at least three different reworks over the lifetime of this
They could make 3 versions of API and adapters that would support drivers written against older versions of API.
I think the real reason is that designing API is hard, and Linux developers have no interest in this. They care only about server platforms and those platforms have open-source drivers because they need Linux. Nobody cares that your USB webcam doesn't work because you don't connect webcam to a server.
I think that Linux could also rely on existing Windows drivers that exist for almost every device. Linux could provide an environment similar to Windows kernel, and run those drivers in a sandbox. For many devices, this is the only sane way to have them working with Linux.
1) Think for a little bit about what you have got and what you want to do
2) Do research. Has this been done before? Have similar things been done before? (Usually the answer is yes to either question)
2) Think whether it's theoretically possible to do what you want to do with the information and parts you have (in this case: there is a working driver for Linux and Windows, but they're closed-source)
3) Fixing computer things usually involves a lot of debugging (tracing). Will tracing help us out here? What tracing tools/skills do you have, what tracing tools/skills would you like to have? Often you'll need to extend the tracing skills/tools you already have, but normally you don't want to spend too much time on this. (In this case, we can trace without any specialized equipment -- we just need a virtual machine.)
4) Once you have the appropriate traces, take a good look at them (you'll probably want some scripting abilities here) and see if you notice any patterns (e.g. "lots of data after this", "just a little data after this", "this bit of data is always the same", etc.)
5) If you don't know what to do at this point, maybe go back to the tracing step