After a grueling two days of kernel muckery, and much help from Mooneer and Mark Williamson (bless you, sir), I finally have a 2.6.24 kernel with OpenVZ and Xen domU. Things I learned along the way:
- Never ever make your generic server in the closet do double-duty as your internet gateway/router. For one thing, internet access is a valuable resource to help diagnose why the dang thing won’t boot. And for another, your roommates hate it when the network is down all day.
- Nobody should ever make kernel patches that start from vanilla (e.g., 2.6.24), add a large feature you can’t get anywhere else (e.g. OpenVZ), and along the way give you lots of unrelated things (e.g., 2.6.24.3). This breaks down the instant two people do it (e.g., Debian), because the unrelated things all conflict.
- The Xen merge into mainline is a good thing, but it’s only a start. I really want: ballooning, PCI sharing, and dom0 support.
- aptitude is actually a great thing. Anybody out there still using apt-get… change your habits. Just start using aptitude. And then, little by little, start figuring out the curses gui and how to mark packages as automatic and how to resolve conflicts and all the excellent stuff aptitude brings. I resisted at first… but it’s definitely worth it.
- Also debian, we really need etc-update.
- It’s very easy to use hilarious amounts of bandwidth when you’re building systems with debootstrap.
- I did something today that made my big domU extremely slow. It seems that under very heavy I/O it just grinds to a halt. I also had very constrained memory and no swap device. So I’m not sure if it just didn’t have space to cache and had to start kicking clean pages, or whether Xen event channel unnecessarily serializes I/O (I’m afraid that it does), or what. DTrace would be sweet.
- I need to get better discipline in system management. I’ve historically used cioppino (the Linux machine) for everything from serving web and subversion, to an irssi/shell server, to hosting my AVR cross-toolchain, to my (excessively convoluted but ultimately perfect for me) email hosting. I need to cut down on what I do with it, and I need to stop installing and building random things that seem neat. Apt makes it so easy, I know… but it becomes an N^2 problem. And I’m absolutely terrified by the number of upgrades aptitude offers… there’s just too much on this system.
Finally, a desperate plea: Can we please have a good cross-platform low-commitment network filesystem?
- NFS is difficult to run safely, hard to expose to temporary untrusted users, and still might require shared UIDs (I think?). Sucks.
- AFP hates symlinks and hardlinks within the shared area, and isn’t available for Windows. Sucks.
- Coda and AFS require lots of setup and aren’t suited for “oh just connect to the server and grab it” use case, with temporary untrusted users. Sucks.
- FTP blows.
- sshfs requires Fuse which concerns me, and requires a shell-equivalent account on the machine. Sucks.
- SMB is well-supported and cross-platform, and doesn’t specifically require an account on the machine. It’s pretty bad at non-FAT32 file properties, though. Permissions and symlinks ehhh sometimes work. Workable.
- WebDAV just seems like a bad solution designed by committee. It kinda works.
Really. That’s it. SMB and WebDAV. This is sort of like the situation with disk filesystems… FAT32 or go home. The least common denominator that Windows supports just seems to win :(
