The Creation of Linux Drivers

[Lost In Cyberia]

Hey everyone. So there's something that's been bugging for a while. I've been thinking about the creation of writing of drivers. (who doesn't think about this in their spare time). I have a ton of questions, and I hope this doesn't cause you guys to skip over answering any of them! Don't feel like you have to answer all 8! I appreciate any feedback. Anyway, what's troubling me is this..

So let's say our company Wintel is about to release a brand new graphics card.

1. What determines if this new card, the Ultron 9000, would work in an OS that has drivers for the earlier Ultron 8000?

2. Let's assume that Wintel's Ultron 9000 is a complete redesign, does Wintel employ it's own software team to write drivers for the new card, to operate with the major Operating Systems like Windows and Apple? Or does Wintel assume that Windows and Apple will have write the drivers for the WIntel card?

3. This seems to be mutually beneficial... Both the hardware vender, Wintel and the OS venders, Windows and Apple both benefit from this. Wintel gets their new hardware on board with the two biggest suppliers of Operating systems. And WIndows and Apple can market their OS to be capable of handling a huge variety of devices out of the box. Anyway the question is, why is it, that some drivers can work on multiple devices? but some drivers can only work on one specific device, but useless for any other device? For example, one of my old HP printers, used a certain device driver. I couldn't get that exact driver, but I got one with a very similar model number and it work well enough. Why/how does this work? Are devices that are made like printers, created on the same "core" hardware and all of the extra 'selling points' and bonus features added on, but don't cause any need for an entirely different driver?

4. For a device to operate on any given computer, is the host OS all it needs to be catered to? What I mean is, do devices rely on any other devices, or is it just the OS that needs to be able to talk one on one to device? I guess what I'm asking is, I should be able to plug my Ultron 9000 card into any computer, and as long as the OS has the driver software for the Ultron 9000, it should in theory be able to interface with it correct?

5. So let's move up a level, to the PC manufacturers themselves. How do they fit into this equation? Most PC manufacturers, like Dell, Acer, HP, etc... they want the license to be able to package Windows and Apple on their hardware... So do they write drivers catered for Windows / Apple / Linux (yea right..) ?

6. Taking that even further, An HP or Acer laptop doesn't create 100% of the of the hardware in their PC, For example, my HP PC comes with an Nvidia Graphics card. And further more... On another computer my MSI graphics card uses Nvidia drivers! (my head hurts) So what's the deal with this? So does MSI buy the drivers from Nividia? I honestly don't know the relationship between how companies create hardware, but have other companies writing the drivers. The same is true for companies like Gigabyte and Evega. They create the card, but have Nvidia supply the drivers? I know between MSI and Nvidia, you can find drivers from each for the same piece of hardware. Does that mean that if MSI creates a new piece of hardware, they give it (or sell?) to Nvidia to write drivers for it. What does Nvidia get from adding their drivers to an MSI card?

7. The same line of questioning goes for not so elaborate pieces of hardware. If a there's something "trivial" like a usb operated mouse. Created by Logitalk. Let's say Logitalk really wants their new usb mouse to work on every system, cause they know that'll be more sales. So do they get programmers to write the drivers for each OS, or is it the big OS' like Microsoft and Apple, who want compatiblity of all devices, so they ask Logitalk for the rights to make a driver for the hardware?

8. Last One!! Okay so to expand on the question above... Let's say that Logitalk does indeed want their device to be used on as many platforms as possible. BUT! Only to those companies who pay them. So how do developers who write drivers for linux operating systems, get the rights to do it? If Logitalk says that their stuff is proprietary, how come we have so many drivers for Intel, Nvidia, AMD chips? Do the companies give out like "Pity" chips that are crappy for linux dev's? Are their legal issues for developers in the linux community to make drivers for software that is held by proprietary companies?

I know this is a massive list of questions...and I don't expect a full response back at once. But if you have any time out there, I'd really appreciate it if you can go over and see what you guys can come up with. These probably aren't really that hard of questions, but it's just been something that's been causing me to scratch my head at.

[cooper]

1. What determines if this new card, the Ultron 9000, would work in an OS that has drivers for the earlier Ultron 8000?

When you design the new card, you are aware of the instructions the old driver is sending to the card. If you can make the new card react identically to those commands as the old card did, you wouldn't need to develop a new driver. But this assumes the new card only does the exact same thing as the old one, just faster. The more common thing you see is that the new card has additional features or instructions that make it really shine. So you'd at least want a driver that also provides those instructions when the situation allows it to benefit from their use.

2. Let's assume that Wintel's Ultron 9000 is a complete redesign, does Wintel employ it's own software team to write drivers for the new card, to operate with the major Operating Systems like Windows and Apple? Or does Wintel assume that Windows and Apple will have write the drivers for the WIntel card?

Windows and Apple will do FUCK ALL for you. There are some standard drivers (VGA, USB...) whose instructions you can engineer your devices to support in a proper manner. For some devices that level of support is all you need. You don't need a new driver for another USB memory stick, harddisk, keyboard or mouse, but if that keyboard or mouse has some funky keys the typical devices don't have you're going to have to provide your own driver to make the OS understand about them. Until this driver is installed, chances are the device will work just fine, except for those funky keys who will just be ignored.

In some cases a manufacturer has created a device that's become MASSIVELY popular. When a new version of the OS comes out and the OS vendor knows the driver model has changed in incompatible ways, they'll actively approach the device maker to make sure their drivers are ready for the new OS, to ease adoption of that new OS. Sometimes the OS vendor even includes this vendor's driver in their OS to make it easier on the user. This benefits both the OS vendor (easy transition to the new OS) and the device vendor (out of the box support for the new OS) so if any money is exchanged for this service, it'll not be a large amount.

3. Why is it, that some drivers can work on multiple devices? but some drivers can only work on one specific device, but useless for any other device? For example, one of my old HP printers, used a certain device driver. I couldn't get that exact driver, but I got one with a very similar model number and it work well enough. Why/how does this work? Are devices that are made like printers, created on the same "core" hardware and all of the extra 'selling points' and bonus features added on, but don't cause any need for an entirely different driver?

Exactly. Old driver knows about the 'normal' instructions for that device range, but if you want the special features (like, say, how to operate the duplexing add-on to the AVH356D model) you're going to need to install the device-specific driver.

Note that hardware vendors prefer to have drivers that support everything within the same range from a single driver because each driver you make demands more time from the driver development team. If you only have 1 driver to maintain, it'll be cheaper.

4. For a device to operate on any given computer, is the host OS all it needs to be catered to? What I mean is, do devices rely on any other devices, or is it just the OS that needs to be able to talk one on one to device? I guess what I'm asking is, I should be able to plug my Ultron 9000 card into any computer, and as long as the OS has the driver software for the Ultron 9000, it should in theory be able to interface with it correct?

The OS is an abstraction to the programs running on it such that the programs don't know the peculiarities of the various drivers. That's the point of the driver model. Your program can say to the OS "I want a triangle here". The OS determines which of the 4 graphics adapters is responsible for that specific screen and then tells the adapter of that screen to draw the triangle. The adapter is effectively the driver but the driver translates the standardized instructions that can be sent to the adapter (DirectX, OpenGL...) to the low-level primitives the card itself operates on (assembly, really) which then makes the card do its magic.

Given that, you'll hopefully understand that the OS actually doesn't have that much to do with the way your card operates. Just with the operations the card can expose to the outside world.

You may now think "But what about new features added after those standardized instructions were set in stone?" Well, if you do some sensible API design, you allow for proper detection of the hardware which would allow any program that wants to to provide specialized support for the funky features of the card. These instructions are at that time not part of the official API so if a different vendor implements something similar chances are your software will need to provide a unique code path within the application to cater to that hardware too. This is why some games work really well on cards by vendor A and only so-so on cards by vendor B. Vendor A was either targeted from the start as the platform to optimize for, or vendor A paid the developer to make this happen. Because here, too, the extra codepath(s) makes for more work by the application developer and so this needs to be compensated for by the device manufacturer.

5. So let's move up a level, to the PC manufacturers themselves. How do they fit into this equation? Most PC manufacturers, like Dell, Acer, HP, etc... they want the license to be able to package Windows and Apple on their hardware... So do they write drivers catered for Windows / Apple / Linux (yea right..) ?

A pure PC manufacturer tries to assemble the cheapest bits of hardware that work with the least amount of hassle. They make money by getting you to purchase the hardware and they lose money by providing excessive support (there's always _some_ support, but if some component is very flaky it will result in a lot of calls so expensive support). This means their challenge is that they want to provide the cheapest possible hardware that still works well. The driver is certainly a component of that, but it's just something that's considered when they work out which hardware they're going to combine. Something else they do is talk to hardware manufacturers where they want X discount on the guarantee that they'll ship at least Y items of hardware but manufacturer has to keep making this exact device for the next Z months.

When you talk about those big 'PC Manufactorers', they often pay some company to not only provide their device for their PC configuration but even employ their brand-specific label. So it's not a Seasoonic power supply anymore, but a Dell power supply. They might even have the hardware vendor develop the driver to include the manufacturer's logo. If you pay for it, they'll do anything.

6. Taking that even further, An HP or Acer laptop doesn't create 100% of the of the hardware in their PC, For example, my HP PC comes with an Nvidia Graphics card. And further more... On another computer my MSI graphics card uses Nvidia drivers! (my head hurts) So what's the deal with this? So does MSI buy the drivers from Nividia? I honestly don't know the relationship between how companies create hardware, but have other companies writing the drivers. The same is true for companies like Gigabyte and Evega. They create the card, but have Nvidia supply the drivers? I know between MSI and Nvidia, you can find drivers from each for the same piece of hardware. Does that mean that if MSI creates a new piece of hardware, they give it (or sell?) to Nvidia to write drivers for it. What does Nvidia get from adding their drivers to an MSI card?

What happens is that NVidia designs the card and then tells hardware makers to actually create a physical card based on that design. Some hardware makers "improve" this design with more elaborate power handling or more memory because marketing says it'll make it sell better. But it's an NVidia chip on a card based on the NVidia reference design. Unless they deliberately did something to prevent it (and there's very little incentive to device manufacturers to do this) the standard NVidia driver will work just fine.

7. The same line of questioning goes for not so elaborate pieces of hardware. If a there's something "trivial" like a usb operated mouse. Created by Logitalk. Let's say Logitalk really wants their new usb mouse to work on every system, cause they know that'll be more sales. So do they get programmers to write the drivers for each OS, or is it the big OS' like Microsoft and Apple, who want compatiblity of all devices, so they ask Logitalk for the rights to make a driver for the hardware?

For common types of devices the OS vendor has already made a driver which very clearly specifies what a device maker should do to make their device compatible with this driver. This means that a device maker now has a choice: Develop your own features that don't fit the existing driver which might give it a unique selling point but a driver to make use of this needs to be developed, increasing cost, or to hell with that feature, and let's work on features that are unrelated to the software functioning of the driver (smoother action, scroll wheel with LEDs inside, that sort of 'feature').

8. Last One!! Okay so to expand on the question above... Let's say that Logitalk does indeed want their device to be used on as many platforms as possible. BUT! Only to those companies who pay them. So how do developers who write drivers for linux operating systems, get the rights to do it? If Logitalk says that their stuff is proprietary, how come we have so many drivers for Intel, Nvidia, AMD chips? Do the companies give out like "Pity" chips that are crappy for linux dev's? Are their legal issues for developers in the linux community to make drivers for software that is held by proprietary companies?

The short answer is that they don't. If a device manufacturer wants to get paid to allow someone else to develop a driver for their device, for free, that device maker can go punch sand. Just buy the device made by their competitor who might be more accommodating. What you typically see is that a driver writer asks for specs of the device. The device maker is HUGELY protective of this (our IP! Our PRECIOUSSSS!) but sometimes a developer is allowed to sign a massively penalizing NDA that would allow that developer to see the specs and develop a driver for it. These specs can't be disclosed but the driver allows you to make some assumptions about how the device works (which is where the danger lies to the hardware maker - looks like you're doing X and I have a patent on that. Pay up, sucka!). The upshot for the device maker is that if your device is now supported under Linux aswell, that's another market you can sell your device to. For many types of devices this is a small market, but if your device is intended for servers (smart card readers, say, to provide the machine with a HSM) this market can be quite significant.

Another trend is that sufficient people get together and pay the maker to develop a driver for their platform of choice. It's not very common, but as always, if you give enough money a business will make it happen. They don't _hate_ your OS, they just don't see the return on investment so prefer to spend their money elsewhere.

[Lost In Cyberia]

Cooper, I should have know you'd respond to this behemoth of a post.. First of all, as always, thank you for your time. This was a huge request. But your answers have satiated my curiosity 10 times over! The only question i'm still a bit confused on is the idea of free open source engineering of proprietary hardware. An example is the FOSS project Nouveau. It's based on reverse engineering of the Nvidia linux drivers that were licensed out. I'm sure nvidia knows that they're doing this, how is it legal?

"The project's goal is to create an open source driver by reverse engineering Nvidia's proprietary Linux drivers."

from:

https://en.wikipedia.org/wiki/Nouveau_%28software%29

Thanks again Cooper!

Funny, related article:

http://www.pcworld.com/article/2911459/why-nvidia-graphics-cards-are-the-worst-for-open-source-but-the-best-for-linux-gaming.html

[cooper]

The legality is quite simple. You're allowed to reverse engineer stuff in all sensible developed countries, which is basically any western country except for the USA. The EFF has a page on the legal issues in the USA.

The process of mostly legal reverse engineering of a proprietary product (hardware) is something like this:

You take 2 teams, A and B. Team A takes the original hardware and works out how it responds to different signals that reach it over its device interface (PCIe/USB/GPIO, whatever). They work out the various states the device can be in, work out how these states are reached and how they react to the various inputs while in those states. Then they write a design document describing all of this. When they're done, Team A is potentially legally fucked - the maker of the original device can sue them claiming Team A somehow had access to confidential documents so they must disappear. Nobody knows who they were and all knowledge of their existence is expunged from any system. Their design document is leaked onto some forum in china or whatever.

Enter Team B. These are genuinely different people from Team A. There *MUST* be ZERO overlap (plausible deniability). They 'discover' Team A's design documents through the magic of the internet. They then develop a new device that adheres religiously to the specifications described in the design document. This device is now a functional and technical copy to the original device, but might still do things slightly different when confronted with the original driver because certain elements of behaviour weren't described sufficiently specifically in the document. During normal operation things are as expected, but in weird edge cases registers contain different default values or certain routines take substantially longer/shorter because certain device optimizations made in the original weren't done for the copy or vice versa.

What you then end up with is the current standoff FTDI has with counterfeiters of their hardware.

In case of the nouveau driver, things are thankfully less complicated. NVidia _wants_ an open source driver, but they worry that people might discover, looking at the driver, what methods and operations are embedded in their device and assume they violated someone's patent. Initially the lawyers called the shots, so all the driver developers could do was look at what the binary blob was doing, working out what these actions did to the hardware and trying to replicate this behaviour in a sensible manner via their driver. This, I might add, with the exception of a developer doing this to circumvent some copy protection scheme (USA! Land of the free, except when it comes to Mickey Mouse!), is entirely legal. Things get murky when you do this with design info in hand after having signed an NDA. If you never signed anything and you never looked at confidential manufacturer documents you know you shouldn't, you're in the clear.

By now some legal issues have mostly worked themselves out and NVidia is occasionally helping the nouveau developers by providing information about the more basic elements of the device to at least reach the level of a working device.

Bottom line is that, for the most part, software development is seen as no different than writing a book. So long as you don't outright plagiarize someone else's work without their permission, it's all good.

[Lost In Cyberia]

Cooper, your devotion to providing quality, well thoughtout answers to our questions is remarkable...You really should start like a blog or compedium of all of the information you provide to the curious minded noobies like us lol. Seriously though I'd hate to think all of this information you take the time to write and think about, going to waste...

Anyway thanks for the answer, that cleared up a lot! You say:

Team A is potentially legally fucked - the maker of the original device can sue them claiming Team A somehow had access to confidential documents so they must disappear

Wow... seriously? This is a lot of shit to go through and do, to make a driver...? It actually is surprising that anyone would go through this and potentially screw up their lives to create a driver. The whole process seems like a very "Don't ask, Don't tell" mentality. Has there been large scale crackdowns by the venders against this?

[cooper]

That example was more for a device itself rather than a driver for it. Did you ever see the movie Paycheck? Ben Affleck plays a reverse engineer that allows clients to clear his brain such that he has no knowledge of ever having worked on the device in question so the client gets a virtual clean-room implementation and he can pass any lie detector/brain scan/what have you that might prove his involvement.

Making drivers is mostly just incredibly hard when you don't have any specs. Typically a device driver writer would sign an NDA with the company in order to get access to specs that would make the writing of that driver a heck of a lot easier. The legal mess here starts when for whatever reason the company decides you didn't follow the NDA and now sues you for breach of contract.

An example of reverse engineering landing a project into trouble is ReactOS , a re-implementation of Windows, that was accused of copying Windows code resulting in a 20-month long audit of the code base.