Joe Auricchio

I work in downtown La Jolla, right opposite the Ferrari/Maserati dealership. AMGs, M-series, and Porsche Ses are everyday sights. Jaguars are like Toyotas. I’m tired of seeing expensive cars. But sometimes there’s something a bit more interesting than usual…

One day on the way to work I saw someone picked up in front of a particularly large house by a chauffeured Maybach. In the words of James L. (wherever you are), “Must be nice…”

A few weeks ago I saw a Ford GT in downtown La Jolla. Really, with gas prices up to $5/gallon, you’re driving one of these? You and Jeremy Clarkson both.

The US Open is in town this week, at Torrey Pines. It’s practically across the street from campus. There are a lot more interesting cars here this week—somehow the AMGs and M-series are even more abundant than usual. I found this BMW Z4 with the German plate “BMW Z4″ nestled behind CVS. An odd place to park what I assume is BMW’s touring show car. As I drove out of the parking lot, I saw a brand new SL63 AMG and a DB9 Vantage.

I don’t say any of this out of envy, merely commentary. I don’t particularly want any of these cars.¹ In fact, I still haven’t figured out what kind of car is right for me. Fast or a smooth ride? Four-door? Hatchback? FWD, RWD? Manual or auto? Low-slung or tall and roomy? I have only the vaguest of preferences.

For now I drive my dad’s trusty old Volvo S70 T5. I say “old” because the car is well on its way to 200,000 miles, but she’s only ten years old, and she looks and feels like a young girl of only 60K.² It’s zippy enough that I can have fun on late night jaunts to campus or wherever; but it’s a Volvo, so it’s comfortable enough that I don’t really mind the six-hour nonstop service from SAN to SBP for holidays and vacations. Plus it has five cylinders, which is just neat. It’s my car and I love it.

Then again, with gas going up and up, these may be the last days of the automobile. Maybe I was born in the wrong decade? Ah well. Bring on the public transit and high-speed rail.

DJ laments there are no practical guides to driver writing.

DJ, allow me, as I so often do, to vocally disagree with you :). You seem to be making two closely related claims:

1. There’s no information in OS textbooks on how to structure a driver to fit into any given operating system’s driver architecture
2. There’s no information in OS textbooks on how drivers ought to interact with their hardware to make useful stuff happen

1. General operating systems textbooks are not the place to find this information, because if they were they wouldn’t be general operating systems textbooks.

Depth and breadth are usually a trade-off in books. If a book were so thorough and detailed in its treatment of Linux drivers that you could use it as your sole reference while writing one, it could not possibly do the same justice to drivers for Solaris, Free/Net/OpenBSD, AIX, Mac OS X/IOKit, Win32, WinNT, Vista, or any other operating system, and it surely can’t cover things like process scheduling, networking, virtual memory, filesystems, or any of those other things kernels sometimes do. The reason you read an operating systems book is to get a (necessarily) broad coverage of what all operating systems do.

If you want to get specific enough to one kernel and one subsystem that you can actually use it as a reference to build a driver, you probably want to put Modern Operating Systems back on the shelf and try Linux Device Drivers, a book focused on exactly what you see as lacking in general operating systems books.

Taking a mild divergence to head off a counterargument, it’s completely acceptable for the usual OS curriculum to exclude drivers, while including many other parts of operating systems, say, basics of concurrency. The difference in curriculum reflects the likelihood the average student will encounter the material during his career. The average programmer has very high odds of using concurrent systems sometime in his career, even more so today with the advent of multicore systems. Any more than the most basic concurrent programming (paranoid locks everywhere) involves some knowledge of what the OS is doing. In contrast, the average programmer will never be anywhere near driver code. Those who do write or maintain drivers take onto themselves responsibility to further research the field, the same way we expect those working with highly parallel systems to study a bit more than locks and semaphores.

2. I’m willing to accept that there’s little information available on actually driving hardware, but that’s because the information you’re looking for is very narrowly applicable, proprietary, distributed across the whole field, in rapid flux, and/or lacks any common principles or organization. Difficult to put that into a textbook with a ten- or twenty-year lifespan.

There’s so little information generally available about how to drive hardware because it’s intensely device-dependent. Once you get to the IO port level everything ends up pretty specific to the particular chip you’re talking to. Sure, ATA’s standard, so it’s not hard to drive a hard disk. But how about the ATA controller chip? How do you ask it to send a command and get the result? Not so standard.

The impression I get from the field (see disclaimer below) is that everybody kinda builds the hw/sw interface to be as simple as possible for the firmware and hardware guys to build. This varies fairly widely with different guys and different organizational knowledge. Firmware and hardware are extremely expensive and difficult to build correctly, and anything that can be simplified should be; anything that can be offloaded to (testable, debuggable, patchable) software should be. When everyone builds their hardware the way easiest for themselves and their organization and offloads everything else to software, everybody’s hardware will look completely different and will have completely different expectations of the driver’s role. Consequently, everybody’s drivers will look completely different.

A secondary reason for lack of standardization is that until recently most first-party drivers for mass-market hardware were closed-source. When your competitors’ specs are closed to you and your own spec is closed to them, standardization is just not possible. This is changing, though, and we’ll see what happens. There have also been notable exceptions, when an entire industry segment decided to stop being dumb and standardize. EHCI stands out as a shining example. (IDE/ATA/SATA as standard disk access might qualify, though they have their origins in ancient disk protocols defined by the operating system, not the disk vendor, and any disk had to meet that protocol or you couldn’t use it. Standardization was imposed by the OS, not agreed upon by the vendors)

In such an environment, OS books can’t really get any more specific than “Here’s how you read from or write to an I/O port” for the same reason they can’t really get any more specific than “Here’s how you read or write to a network socket”. To tell you how to use, say, FTP, is to tie themselves to today’s practice and take up paper and author’s time that could be better devoted to general principles of networking. They probably ought to describe PIO and DMA, for the same reason they describe IP and TCP. But the OS books are and should be mute on how to drive the hardware. Those rules are different between every chipmaker and every chip, and change every year and every rev. They belong in datasheets and reference manuals, not general textbooks.

Conclusion

For now, every operating system has a different driver architecture, so a general operating systems textbook can’t and shouldn’t devote too much attention to any given system. Furthermore, every piece of hardware has a different host interface, so a general operating systems textbook can’t and a driver reference book sometimes shouldn’t devote too much attention to any given piece of hardware.

The way to change this is for different kernels to agree on a driver architecture, or for different hardware makers to agree on a host interface. The first is not likely to change anytime soon; Linus is not about to chuck the bottom quarter of 2.6, Apple’s kernel team hasn’t shown signs of life lately, Sun still hates everyone, and Microsoft is still dumb.

What is changing is that host interfaces are standardizing. USB was a great leap forward in this regard. Other than the UHCI/OHCI misstep, EHCI means anybody’s host adapter works the same as any others, and device classes mean anybody’s device works the same as nay others. No more does every serial and parallel widget in the store need its own driver. Disk interfaces have long been standardized (out of necessity). I thought Ethernet NICs have a broadly-supported standard, but I can’t track it down.

Disclaimer: I’m very new to hardware design, I’ve never written a driver, and I’ve only been through the first-level undergrad operating systems course. I calls ‘em as I sees ‘em, but I may not see ‘em very well from where I stand. I welcome discussion.

Today, Kyle said to me:

So, you know that whole hd dvd key thing? Everyone and their grandmother knows it now, or knows where to find it. But I don’t think anyone knows what the heck to do with it.

Why did the HD-DVD key become such an issue?

Content providers: Takedown letters are not the way to stop piracy. Still less are they the way to squelch people with an agenda.

Bloggers, Diggers, etc: Copying and pasting a couple dozen bytes does not equate to taking to the streets in glorious revolution. The key’s been public for months. Those who needed it to do business (software developers and pirates) had it long ago.

Forgive me if I sound reactionary. I’m on the bloggers’ side on the whole. Certainly I’m no friend to the MPAA and RIAA. I just thought we’d been through all of this back in 2002 when DVD-Jon went to trial. It’s the same DMCA, same technology (video DRM), same fair use and anticircumvention issues. Of course, none of these issues actually came up in Johansen’s trial; they were just discussed to death on the net. I’m annoyed that this all came back a few days ago, but glad everybody shut up at the end of the day.

TMBO had a photo (sorry, no link) of a guy who tattooed the key to his chest. He’s going to feel really silly next year when nobody remembers or cares anymore, and even sillier in five when the next-next-gen formats arrive with the same folks backing them.

(I hope he feels silly already because back when we had the export restrictions fifteen years ago people did the same with strong crypto algorithms.)

Speakeasy, the last insanely great national ISP, just got bought out by Best Buy.

Speakeasy is known in the community for giving the speeds they promise, not throttling on the last mile nor unreasonably overselling their middle mile, for not blocking any inbound or outbound traffic, and encouraging users to run personal and game servers. These are things no other ISP really does. We’d be with them today, on 6×768 DSL, if we were closer to our CO. I can personally attest to their excellent customer support: the couple of problems I had were both resolved in a ten minute phone call to actual employees (not call centers). Not bad for a company of only 300 people.

And now they’ve been bought out by Best Buy for barely a year’s revenue.

At least it’s the Best Buy business division. That means we won’t have salesdrones pressing DSL on top of service agreements. And business-facing people seem to have their stuff together, on the whole. So I’m not completely without hope. I guess we’ll know in a year or so if Best Buy nukes the management.

Shoutouts to (in no particular order):

Paul, Ava, Ben, Kelsey, Rushi, Sheenika, Jonathan, Dan, Chris, Mooneer, Stephi, Hanna, Scott, Danny, David, Nik, Jess, Brad, Bethany, D.J., Sheree, Aaron, Lily, Eric G, Auston, Mark “The Knife” “The River” Gahagan, Meg, Chi, Amrit, Tam, Amado, Rego, James, Kim, Amy, Proud, Sharon, Christine, Michael, Melissa, Christie, Eric B-D, Billy, Cody, Dylan, Brandon, Iron Jeff, The Swede, Ryan, Cano, KG, Gabe, Cha-cha, Mary (my other mother), Bilow, Flo (”zis country would be so much betteir if it weir cohvered wis a meeteir of snow!”), Kris, Amanda, Lizzie, Rose, Brigette, Kerry, Marla, Kristjiana, Iain, Mike, Van, everyone else from HS, John and Elaine, Brian, Chris, Ted, Danny, everyone else from UCSD, Bill Clabby, Jeanette Ibarra, Professors Chu, Ord, Tullsen, Sato, Ito, Pineda, and Rhodes, Gabriele, Pat, Matsumiya-sensei (wherever you are), the Big Domy (you know who the Big Domy is), and of course Teresa, Rick, Amy, and Jessie. If, somehow, I forgot you, you know you’re on there too.

Further corporate propers to:

Apple Core OS team, 2 Dogs Coffee (Morro Bay), Eventful, Fern Canyon Press (and all their endeavours), Microsoft Xbox 360 team (all right, ya done good), Nintendo, Xilinx (I hate your software but I love your hardware), most of UCSD, Google, all the backbone network operators, OmniGroup, Matz, _why, and the Ruby core team, Linux kernel developers, Red Hat/Fedora, every single man woman and child who has contributed to Debian, NoMachine, irssi.org, Freenode, GNU Project, SureFire, Streamlight, Wes Hinkle’s San Diego Volvo (Service department), and of course Apple circa 1976, for the dream and the spirit.

All of these people touched me in 2006, almost universally for the better, and made my life what it was. I look forward seeing you all in 2007.

My previous post pointed out to me a failing of terminology to describe compatibility. To paraphrase FOLDOC:

• Backward/downward compatibility means playing DVD discs in Blu-Ray players. It is a property of the new system.
• Forward/upward compatibility means DVD players were originally designed to play Blu-Ray discs with some level of degradation. It is a property of the old system. (This is not true, of course. DVD players have no idea what a Blu-Ray disc is and will spit it right out. The physical layer wasn’t designed to be gracefully forward compatible.)

I’m looking for a term that means that Blu-Ray discs were designed to be played in DVD players. It’s a property of the new system; the new system was built to work with the old system, the old system never had a clue about the new system.

Via Bruce Schneier’s blog. Read it first.

When you reduce “admissibility” to a more simple and general form, it becomes the question “What happens to the data after it changes hands from the provider to the user?”. At some point, the handoff must occur, and beyond that point the server can have no knowledge or influence on the data’s further disposition. A few technologies, such as end-to-end encryption (SSL, etc) can help move the handoff to the client endpoint’s network stack, but at some point the data is decrypted, and it becomes accessible to the user’s code and eyeballs. This is the intention of a good secure system — to give the right data to the right users.

With that, necessarily, is the risk that other code and eyeballs will steal that data from the intended recipient - and that is a completely separate security problem that can only be solved on the client side. The parameters of the problem are the same “Four As” (or “CIA”) we already know. The security of the entire system scales like a fractal, to smaller and smaller scopes, until at last we have CRT to eyeball and fingers to keyboard (which leaves the realm of computer security and becomes an old-fashioned physical security problem) ((I’m not trying to minimize the value of physical security; if anything, it’s a lot harder than computer security. It’s merely a different kind of problem, and a different set of people need to work on it once it reaches that level.))

The security of the client is a problem; but it is neither the server’s problem nor a problem the server can solve at all. A server can never have any assurance whatsoever that its client is not vulnerable to attack. There are two ways for the server to obtain an answer to that question: the server can invasively investigate the client, or the client can claim to the server that it is secure. If the server invasively investigates the client, it can be tricked into investigating a clean-looking virtual machine. If the client claims it is secure, or rather, that some sort of configuration-management software running on the client claims the client is secure, that response can be spoofed by an attacker, or the configuration-management software can be compromised.

If this whole server-trusting-the-client’s-configuration sounds vaguely familiar, it’s exactly the same problem that ruins identd: the trust isn’t placed in identd, the trust is placed on the sysadmin of the remote machine. But how do you know to trust them? How do you know their machine is not lying and has not been hijacked? You cannot, and that is why identd is not a usable authentication system. ((So why do most IRC networks still require it, or make you wait thirty seconds until the request times out?))

Ultimately, the admissibility question reduces to “What are you going to do with this data after I give it to you?” This is not a new question, nor is it an easy one to ask and answer. Some of its unhappier answers include users taking laptops home after work where they can be burgled, or users deliberately giving or selling data to third parties. These are all completely outside the design of the security system. ((This problem of “what are you going to do with the data, now that you have it?” applies at all levels, including within the server as requests are processed, an area I think may be overlooked too often. This is the level at which you ask questions like “Do I trust that this data will not be snooped out of the kernel’s networking stack, before it’s sent out the ethernet port?”—don’t laugh, it’s happened!))

Until the day when data itself grows teeth and can defend itself against misuse, we have to carefully limit who we authorize, do our best to teach how to secure their own environments, and finally, trust those authorized users and their environments. That’s the way we do things now, and it’s not always a very fun way (especially that second part), but it’s the only way.

I hope somebody’s working on that teeth thing, though.

A final word: One-time passwords and time-based credentials (Opie, SecurID…), properly implemented, can make all forms of sniffing a LOT less profitable. The first credit card company that uses rotating card numbers based on a SecurID-style token is going to get my business. Alternatively, I’ve heard some European banks give customers scratch-and-sniff cards with a few dozen one-transaction authorization codes. That’s a nice low-tech solution that’s just as good at the end of the day.

On the SAT test, nigh on two years ago, I had to read and answer questions about a lovely little essay on the subject of immortality. It fascinated me, and when I got home after the test, I tracked it down. It was an op-ed piece in The New York Times, entitled “A Brief Version of Time” written by Alan Lightman in 1993.

Inspired by Jess’s latest blog entry, I dug it up again. The only clean copy I could find was in Google’s cache of an old mailing list archive that has been taken down. I <3 the internet. (There is, of course, a perfectly good copy in Nexis somewhere, and since I’m on campus I have access to it, but Nexis is scary and I don’t know how to use it. I know, I’m terrible. I should go to one of the “How to do research that doesn’t start at Google and end at Wikipedia” workshops at the Library.)

Here it is, for all the world to read again, until the Times finds me.

A Brief Version of Time, by Alan Lightman.

The most sensible kanji ever

concave, hollowed out, sunken in: ?

convex, protruding: ?

The least sensible kanji ever

circle: ?

round: ?

Once again, I’ve procrastinated myself into a terrible place. I have a Japanese midterm at 10am this morning, with part two on Tuesday afternoon. I’ve put maybe five hours of study into Japanese this quarter, four of them tonight.

I have to learn four grammatical constructs. Two of them—honorific language and humble language—are similar in mode of operation: modifications to verbs based on the subject’s relation to the speaker. The other two are also similar—postfixes to verbs with only one character difference between them. Speaking as a cognitive scientist in training, their similarity means it’s easy to confuse them without extensive practice. That means I’m screwed.

I have to learn a few dozen vocabulary words. We’ve had three vocab quizzes on subsets of the list. I missed one quiz outright. For the other two, I “studied” by cramming the night before and morning of the quizzes. Without repeated practice, I have no actual retention of the words. So I’m starting from scratch again now. After the midterm, I still won’t remember them, which will burn me again on the final, and burn the worst someday in the future when I have to actually use the words in conversation.

I have to learn thirty kanji: their pronunciation, words containing them, and their English meanings. Thanks to StudyCard Studio at least I can drill these easily. But twelve hours is just not enough time to learn.

Oh, and I have to do listening. This is probably my worst area. Since I rarely watch anime (FMA at Scott’s is the first I’ve seen in quantity since… May?), the only listening practice I have is an hour a day in class. Even then, I’ve noticed I’m no longer listening, I’m only hearing a few words and guessing at the meaning. I’m sure there are first-year Japanese students who can rock me at listening. Cogsci time again: listening and understanding a language is something only practice can bring. Unlike math, heroic mental effort will not help you. Knowledge of writing and reading will not help you—they are completely separate skills. The brain needs practice hearing and interpreting the language. Practice I lack.

I shouldn’t be in Japanese 20B. Christ, I never learned 10B properly. The difference is, the ten series was an easy A or a zero-work B. The twenty series is not kidding around anymore. It’s too late to switch to P/NP this quarter. I’m already down in points thanks to missing a quiz and a few lectures.

On top of all this, I don’t know if I can even go to Japan fourth year. I don’t know if I can even responsibly continue taking Japanese—I need all the classes I can take. I’m already looking at a solid six years. Japanese might mean a major sacrifice… such as a minor in CS (which I’m seriously considering). Perhaps I can’t do it at all. I need to figure this all out, and soon.

But instead of, gee, I don’t know, studying, I’m writing this blog post. I’m listening to the Lovedrug CD I got at the concert yesterday (I should have been studying last night, too). I’m fighting with AppleRecords.jar to download as much music as possible from shared iTunes. I’m talking with friends on Jabber.

God I’m an idiot.

Jump to Step 7 and run till completion.