To be able to change BIOS settings from the command-line on a Dell Poweredge, you need the syscfg utility. It’s very useful when you want to change a configuration on, for example, 32 nodes at once, without having to plug screen, plug keyboard, reboot, change setting, reboot 32 times. Here is how I installed it [...]
I had, until recently, a problem when it came to test memory on the nodes in my lab. Until now, I was able to PXE boot memtest+, but had to go down to the lab and plug a screen to check the output. Multiple annoyances: first I had to move my ass to the lab [...]
Buy one 1U server from $supplier, specifically ask for a pair of rails, learn that “Of course it comes with rails!”.
Two weeks later. Buy a 42U rack, and eight 1U servers, all of these from the same $supplier, and at the same time. Receive your rack and 8 servers, without rails. Inquire by email: “No, [...]
Take one 3U server from SuperMicro, fitted with an Adaptec 31605 SATA RAID controller and 16 one-terabyte disks. Create a RAID 0+1 array for data storage, with 14 of these 16 disks. You get:
#available storage
[root@sam119 ~]# df -h /data
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 6.3T 941M 6.0T 1% /data
#write test
[root@sam119 ~]# dd if=/dev/zero of=/data/toto.txt [...]
Actually, this is not RPM Hell but rather crappy packaging hell:
[root@sam70 ~]# rpmbuild –rebuild xen-3.1.0-10.fc8.src.rpm
Installing xen-3.1.0-10.fc8.src.rpm
error: Failed build dependencies:
transfig is needed by xen-3.1.0-10.x86_64
libidn-devel is needed by xen-3.1.0-10.x86_64
texi2html is needed by xen-3.1.0-10.x86_64
SDL-devel is needed by xen-3.1.0-10.x86_64
curl-devel is needed by xen-3.1.0-10.x86_64
libX11-devel is needed by xen-3.1.0-10.x86_64
python-devel is needed by xen-3.1.0-10.x86_64
ghostscript is needed by xen-3.1.0-10.x86_64
tetex-latex is needed by xen-3.1.0-10.x86_64
gtk2-devel [...]
I can’t resist sharing a few pictures of my lab at work:
Total: 100 nodes…
Suddenly getting 60% packet loss between the LAN and the cluster, just because
kernel: ip_conntrack: table full, dropping packet.
Fix the problem by raising the limit from 65536 to 2097152 slots. If this limit is reached, that’ll eat 700MB of memory, ouch.
Lose one hour thinking this RAID controller driver must be buggy because “mount /dev/sda1 /mnt/sda1” errors out with “No such device“. After this hour, figure out from man 2 mount that ENODEV means “filesystemtype not configured in the kernel.” instead. Duh.
Accept the fact that some gigabit switches from an anonymous vendor that has a name [...]