Archive for the 'Sysadmin' Category

Stuff that happens to sysadmins

mercredi, mars 5th, 2008

Buy one 1U server from $supplier, specifically ask for a pair of rails, learn that « Of course it comes with rails! ».

Two weeks later. Buy a 42U rack, and eight 1U servers, all of these from the same $supplier, and at the same time. Receive your rack and 8 servers, without rails. Inquire by email: « No, servers don’t come automatically with rails, you didn’t ask for them ».

Thanks, Dell. It’s always a pleasure.

Geek pr0n again

mardi, janvier 22nd, 2008

Take one 3U server from SuperMicro, fitted with an Adaptec 31605 SATA RAID controller and 16 one-terabyte disks. Create a RAID 0+1 array for data storage, with 14 of these 16 disks. You get:

#available storage
[root@sam119 ~]# df -h /data
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 6.3T 941M 6.0T 1% /data

#write test
[root@sam119 ~]# dd if=/dev/zero of=/data/toto.txt bs=4096 count=2621440
2621440+0 records in
2621440+0 records out
10737418240 bytes (11 GB) copied, 38.8372 seconds, 276 MB/s

#read test
[root@sam119 ~]# dd if=/data/toto.txt of=/dev/null
20971520+0 records in
20971520+0 records out
10737418240 bytes (11 GB) copied, 31.7011 seconds, 339 MB/s

(See also the previous Geek pr0n entry, complete with JPEGs!)

RPM Hell

lundi, octobre 15th, 2007

Actually, this is not RPM Hell but rather crappy packaging hell:

[root@sam70 ~]# rpmbuild –rebuild xen-3.1.0-10.fc8.src.rpm
Installing xen-3.1.0-10.fc8.src.rpm
error: Failed build dependencies:
transfig is needed by xen-3.1.0-10.x86_64
libidn-devel is needed by xen-3.1.0-10.x86_64
texi2html is needed by xen-3.1.0-10.x86_64
SDL-devel is needed by xen-3.1.0-10.x86_64
curl-devel is needed by xen-3.1.0-10.x86_64
libX11-devel is needed by xen-3.1.0-10.x86_64
python-devel is needed by xen-3.1.0-10.x86_64
ghostscript is needed by xen-3.1.0-10.x86_64
tetex-latex is needed by xen-3.1.0-10.x86_64
gtk2-devel is needed by xen-3.1.0-10.x86_64
libaio-devel is needed by xen-3.1.0-10.x86_64
/usr/include/gnu/stubs-32.h is needed by xen-3.1.0-10.x86_64
dev86 is needed by xen-3.1.0-10.x86_64
gnutls-devel is needed by xen-3.1.0-10.x86_64
openssl-devel is needed by xen-3.1.0-10.x86_64

Picking only one example from this list: why on earth does xen need gtk2-devel?

Let’s just grin, and do as requested, after all, it’s not as if I had a choice. After figuring out that /usr/include/gnu/stubs-32.h is actually provided by glibc-devel (and the src.rpm building machine didn’t know it…)

[root@sam70 ~]# yum install transfig libidn-devel texi2html SDL-devel curl-devel libX11-devel python-devel ghostscript tetex-latex gtk2-devel libaio-devel glibc-devel dev86 gnutls-devel openssl-devel
Loading « installonlyn » plugin
Setting up Install Process
Setting up repositories
base 100% |=========================| 1.1 kB 00:00
updates 100% |=========================| 951 B 00:00
addons 100% |=========================| 951 B 00:00
extras 100% |=========================| 1.1 kB 00:00
Reading repository metadata in from local files
primary.xml.gz 100% |=========================| 341 kB 00:01
updates : ################################################## 817/817
Added 12 new packages, deleted 0 old in 1.03 seconds

Transaction Summary
Install 76 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

Total download size: 116 M
Is this ok [y/N]: y

Now everything’s downloaded…

Transaction Check Error:
file /usr/share/man/man1/asn1parse.1ssl.gz from install of openssl-0.9.8b-8.3.el5 conflicts with file from package openssl-0.9.8b-8.3.el5
file /usr/share/man/man1/nseq.1ssl.gz from install of openssl-0.9.8b-8.3.el5 conflicts with file from package openssl-0.9.8b-8.3.el5
file /usr/share/man/man1/ocsp.1ssl.gz from install of openssl-0.9.8b-8.3.el5 conflicts with file from package openssl-0.9.8b-8.3.el5
file /usr/share/man/man1/smime.1ssl.gz from install of openssl-0.9.8b-8.3.el5 conflicts with file from package openssl-0.9.8b-8.3.el5

That’s actually due to openssl-devel package pulling in i386 and x86_64 versions of the openssl package, which happily conflict due to both of them shipping… the same manpages! I wonder how they manage to get automated builds working. (Solving that: rpm -ivh –force –nodeps /var/cache/yum/base/packages/openssl-* . Doesn’t look like chainsaw work at all…)

As a bonus, note how the error message doesn’t indicate at all that the two conflicting packages, even if they’re named the same, are not actually the same, given both of them are for different architectures.

Sigh. Thanks, RedHat.

Geek pr0n

mardi, juin 19th, 2007

I can’t resist sharing a few pictures of my lab at work:

Total: 100 nodes…

Stuff that happens to cluster sysadmins

vendredi, février 9th, 2007

Suddenly getting 60% packet loss between the LAN and the cluster, just because

kernel: ip_conntrack: table full, dropping packet.

Fix the problem by raising the limit from 65536 to 2097152 slots. If this limit is reached, that’ll eat 700MB of memory, ouch.

news for few, stuff no-one cares about