Stuff that happens to cluster sysadmins

Suddenly getting 60% packet loss between the LAN and the cluster, just because

kernel: ip_conntrack: table full, dropping packet.

Fix the problem by raising the limit from 65536 to 2097152 slots. If this limit is reached, that’ll eat 700MB of memory, ouch.

3 Responses to “Stuff that happens to cluster sysadmins”

  1. Pierre Phaneuf Says:

    Hey! That’s work stuff!

  2. Hastur Says:

    You might want to consider setting ip_conntrack_tcp_timeouts to more reasonable values than the defaults

  3. Calim (Martin H.) Says:

    Hi Colin,

    It happened to me too :) Remember that /proc/sys/net/ipv4/netfilter/ip_conntrack_max is reseted do it’s default value each time ip_conntrack kernel modules in reloaded. It happens on RedHat when issuing a “service iptables restart” for examples.

    Solution:
    adding args to the module loading statement in modprobe.conf (redhat) , like this:
    options ip_conntrack hashsize=32768

    With ip_conntrack_max = hashsize * 8

    Greets !!!

Leave a Reply

news for few, stuff no-one cares about