Jared

I have just had my first successful boot of a root filesystem completely contained in RAM using unionfs and squashfs. I used PXELinux to boot the second system. The second system downloaded the kernel and the initrd. The boot sequence began and the initrd downloaded the squashfs root filesystem and placed it on a ram disk. The initrd mounted the squashfs, then mounted a tmpfs which would act as the read/write portion of the unionfs. The squashfs and tmps were mounted into the unionfs. The initrd then called pivot_root to change to the unionfs as the root and began the init sequence of the squashfs.

The only downside was that it took about 3 1/2 minutes for the 95MB squashfs to download, but I think this was because the test was run under two virtual machines and networking for VMs is not always the best. With this complete I can now work on getting it setup to be a webserver and get the LVS setup working.

P.S. I think I might also create a Ubuntu Live CD that will load the filesystem into memory and run from there. Should result in a much quicker operating system.

I feel quite sheepish. In the previous post I described a problem of the Ubuntu 5.10 kernel booting the initrd, but the 6.06 kernel not booting it. Well after a break of a couple of days from the problem, I found the answer in under 10 minutes. the 5.10 kernel has ext2 support built into the kernel whereas the 6.06 has it built as a module. A recompile of the kernel with ext2 support built in allowed the system to boot.

After talking with my friend Andrew, I decided to look into booting a more or less standard initrd that downloaded the real rootfs and mounted that. This is when I came across squashfs. Squashfs creates a compressed readonly filesystem which is fast and compact. In conjunction with unionfs, it is possible to use squashfs as the rootfs by creating a ram disk for the purpose of storing the changes that will happen on the read only squashfs. This will provide for a smaller memory footprint allowing for less memory requirements and/or more available applications.

As a sidenote, I've done a lot of reading and searching on the Internet about diskless nodes in cluster settings, and it seems that most if not all have a dependence on a file server for the node operation. While the overall design of the web cluster called for an external storage solution, which would hold the actual website content, it would be possible to place the website within the squashfs image, thereby removing the dependence on an outside source for operation. 

In my previous post I discussed my plans of designing an easily scaled cluster. So far I've created an initrd of Ubuntu 6.06 running lighttpd and php5. The uncompressed size of the initrd is about 200MB. I've been able to load and run a system with this initrd, but have only had success with the stock 5.10 kernel. The 6.06 kernel panics with not being able to mount the rootfs. 

Tonight I attended a meeting of the BYU UUG about LVS. The focus of this presentation given by Lloyd Brown was an introduction to LVS and it's potential. The environment in which Lloyd uses LVS is quite simple and basic, but effective. His purpose was to balance the load of SSH across multiple computers with one entry point, but not concerned with high availability. LVS has one weakness (on its own). The setup calls for one director and only one director. That means that if the director goes down, it all goes down. But thanks to Linux-High Availability it is possible to set LVS up in such a way that if the main director went down, a backup can take over on the fly.

With that in mind, I was thinking of expanding on that idea. What if one used LVS-NAT for the configuration and setup the nodes (real servers) to get their configuration and filesystem via PXE? I know it is possible to boot Linux in such a way, but admittedly haven't done it before. The reason I'd want to have it PXE boot is because it would then be a simple matter to keep say a website up-to-date on all the nodes, and it is very simple to expand the cluster by simply getting a box and adding booting it. Using the heartbeat method of high availability, one would have a cluster that is highly scalable, easily scalable, and highly available.

When dealing with a handful of nodes this would most likely be overkill, but if one was dealing with 15-??? nodes, this would almost be necessary. My first step will be to PXE boot Linux by mounting the filesystem via NFS. Then I'll setup LVS for a simple website. Finally, I'll setup the heartbeat for high availability