Admin sys

Stuff that happens to cluster sysadmins

Suddenly getting 60% packet loss between the LAN and the cluster, just because kernel: ip_conntrack: table full, dropping packet. Fix the problem by raising the limit from 65536 to 2097152 slots. If this limit is reached, that’ll eat 700MB of memory, ouch.

Continue reading...

Stuff that happens to sysadmins

Lose one hour thinking this RAID controller driver must be buggy because « mount /dev/sda1 /mnt/sda1 » errors out with « No such device« . After this hour, figure out from man 2 mount that ENODEV means « filesystemtype not configured in the kernel. » instead. Duh. Accept the fact that some gigabit switches from...

Continue reading...