Howto: Set up a Gentoo LDOM

'''WARNING: I'm pretty sure this doesn't actually work. We've mostly given up on Gentoo LDOMs until sunvdc can stop being retarded. --Steven Godofsky 15:16, 13 May 2009 (EDT)'''

LDOMs using anything other than Solaris are a bit iffy: *BSD support appears to be almost totally nonexistent for Niagara (UltraSPARC T1/T2/T2+) chips and Linux has its quirks. Nonetheless it is possible to get Gentoo onto an LDOM, although Google appears to think that we're the first to do this.

Kernel Modules and Drivers
The biggest issue with Linux LDOMs is the same as the biggest issue with Linux/hardware interoperability in general: Drivers! Those dastardly things that just never seem to work! Well, they sorta work for LDOMs, but the problem is nobody seems to have bothered to include them in live CDs. The special drivers needed are:


 * sunvdc
 * sunvnet

These two drivers enable support for the virtual disks and virtual net interfaces used by LDOMs, respectively. Also, you might need to build in LDOM support—I'm not sure if that's only needed for the server, or needed for the LDOM itself as well.

Getting a bootable environment with the correct drivers
The Debian install disk for SPARC has the right drivers but in order to load them you have to escape to the shell and modprobe them manually. Furthermore it appears that the drivers on the Debian disk are glitched to hell and back: every time I tried them, biblical levels of corruption ensued. In order to get a working version, we took another SPARC environment (Shakuru) and installed Gentoo on it. So, step one: Find a SPARC box and put Gentoo on it bare-metal. I'm not going into how to install Gentoo, or the specifics of installing it on SPARC; there's a fair amount of documentation out there already. One note: be sure that your mcpu flag in etc/make.conf is either unset or, if you're on a system with the same processor type (ultrasparc, etc.) as your LDOM server, you can set it to your processor type. If you have it set to something else, things don't work right.

Once Gentoo is installed, recompile your kernel with sunvdc and sunvnet installed. Beware: the kernel has a size limit of 7.5MB, and the SPARC kernel is about three times larger than the x86 kernel for some reason. So don't compile in any modules that aren't either necessary or immensely helpful to have in the kernel directly.

Anything you install at this point will go into the LDOM. What you are creating is the basis for a |Stage4 tarball. Keep this in mind: either go for a minimalist system or include all of the packages you want on the final system. Your choice. Chances are your T2 is much faster than your spare SPARC host, however. Therefore bear in mind which host will compile stuff faster before you unleash the emerge cannon of gcc output spammage.

Once you've got all the stuff you want in your environment, follow the steps listed at this very helpful Gentoo Wiki page (which I've already linked above): http://en.gentoo-wiki.com/wiki/Custom_Stage4

Below this point, working status is uncertain
dd /dev/zero into a file, and make the bs/count multiply to yield the size you desire for the LDOM's primary disk. fdisk the file. Create a sun disklabel, specifying 1 head, 1 cylinder, 1 sector/cylinder. Create two partitions: root and swap. You can forgo swap if you wish. Then format the root partition to your desired filesystem. Loopmount (mount -o loop) the file to some mountpoint of your choosing, and untar the stage4 you created using the process linked above into the root partition.

Once this is done, be sure to edit /boot/silo.conf and /etc/fstab such that they reference not sd[a-z][0-9], but vdisk[a-z][0-9]. Otherwise your system will fail to boot.

I'm assuming you know how to set up an LDOM up until the point when you're connecting to the console and actually booting it. That process should be completed. If it isn't, do it now. Go on. Do it!

With silo.conf and fstab having been fixed, copy the image over to the LDOM host and add it to the virtual disk server, then the LDOM. If the disk and partition specified in /etc/fstab is /dev/vdiska1, make sure that the file is the first disk assigned to the LDOM. If your root partition is to be on /dev/vdiskb1, then make sure that the file is the second disk. And so on...

Boot your environment
Bind and start the LDOM. Then connect to the console. Boot to the correct diskfile. When Silo loads it should boot automatically but if it doesn't just enter "gentoo" or "gentoo nox" at the prompt. You should know what these do from your installation on the original SPARC bare-metal host.

If you did everything right, it should start. I think.

Current status: we've got it booting. But there might be some driver issue—we have a series of I/O errors, and all kinds of disk corruption reported. We have no idea what causes this, but...IT BOOTS!