Tikejhya: Ashish Nepal

Knowledgebase

Category: IPtables

kernel: nf_conntrack: table full, dropping packet

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

Aug 7 12:16:17 kernel: nf_conntrack: table full, dropping packet.

What is nf_conntrack?

The Linux Netfilter system (commonly known as the iptables firewall) includes a facility to track connections provided by the nf_conntrack kernel module. The state of a connection is tracked to allow an efficient traversal through the Netfilter firewall tables, as well as to provide the ability to filter based on detailed state of a connection.

What is ip_conntrack?

The Linux Netfilter system (commonly known as the iptables firewall) includes a facility to track connections provided by the ip_conntrack kernel module. The state of a connection is tracked to allow an efficient traversal through the Netfilter firewall tables, as well as to provide the ability to filter based on detailed state of a connection.

Reason?
Heavy traffic (could be D/DOS, or less expected traffic or probably left default without system tweak )

# Check Current Count Value
sysctl net.netfilter.nf_conntrack_count
# Max supported value
sysctl net.netfilter.nf_conntrack_max

# Netstat count
netstat -tn | awk '{n[$6]++} END { for(k in n) { print k, n[k]; }}'

# conntrack timeout value
sysctl -a | grep conntrack | grep timeout

# Increase conn track limit
/sbin/sysctl -w net.netfilter.nf_conntrack_max = 196608

Permament change@
echo net.ipv4.netfilter.ip_conntrack_max = 196608 >> /etc/sysctl.conf

# Heavy traffic tuning
echo "120" > /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_time_wait

Howto what is max nf_conntrack assigned?
# /sbin/sysctl -a | grep -i nf_conntrack_max
net.netfilter.nf_conntrack_max = 65536
net.nf_conntrack_max = 65536

Howto: what is current nf_conntrack_count?
linux:~# /sbin/sysctl net.netfilter.nf_conntrack_count
net.netfilter.nf_conntrack_count = 12742

Modules Related

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

/sbin/lsmod | grep -i ‘ip_tables|conntrack’
nf_conntrack_ipv4 10346 3 iptable_nat,nf_nat
nf_conntrack 60975 4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4
nf_defrag_ipv4 1073 1 nf_conntrack_ipv4
ip_tables 9899 2 iptable_nat,iptable_filter
x_tables 14175 3 ipt_MASQUERADE,iptable_nat,ip_tables

Remove nf_conntrack if no NAT is required.

/sbin/rmmod iptable_nat
/sbin/rmmod ipt_MASQUERADE
/sbin/rmmod rmmod nf_nat
/sbin/rmmod rmmod nf_conntrack_ipv4
/sbin/rmmod nf_conntrack
/sbin/rmmod nf_defrag_ipv4

It could be simply activated by using command:
iptables -t nat -L -n

Howto Increase value of nf_conntrack?
# sysctl -w net.netfilter.nf_conntrack_max=131072

This must not be altertered unless you are 100% sure what you are doing. And that is not done yet, there is direct relation with hash-table.

# Before Increasing anything i would suggest to decrease these value,
echo "86400" > /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established
echo "90" > /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_time_wait

This has to be followed by increasing the size of nf_conntrack hash-table

echo “nf_conntrac_max/4” >> /sys/module/nf_conntrack/parameters/hashsize

So in this case,

# echo “32768” > /sys/module/nf_conntrack/parameters/hashsize

NOTE: This is critical, and this should not be altered if you donot know what you are doing, and if you donot know iptables/conntrack/nat.

Real time logfile:
netfilter]# ls -al /proc/sys/net/netfilter/nf_log/
total 0
dr-xr-xr-x 0 root root 0 Jul 8 10:16 .
dr-xr-xr-x 0 root root 0 Jul 8 10:16 ..
-rw-r–r– 1 root root 0 Jul 8 10:16 0
-rw-r–r– 1 root root 0 Jul 8 10:16 1
-rw-r–r– 1 root root 0 Jul 8 10:16 10
-rw-r–r– 1 root root 0 Jul 8 10:16 11
-rw-r–r– 1 root root 0 Jul 8 10:16 12
-rw-r–r– 1 root root 0 Jul 8 10:16 2
-rw-r–r– 1 root root 0 Jul 8 10:16 3
-rw-r–r– 1 root root 0 Jul 8 10:16 4
-rw-r–r– 1 root root 0 Jul 8 10:16 5
-rw-r–r– 1 root root 0 Jul 8 10:16 6
-rw-r–r– 1 root root 0 Jul 8 10:16 7
-rw-r–r– 1 root root 0 Jul 8 10:16 8
-rw-r–r– 1 root root 0 Jul 8 10:16 9

Other Internet Material, that i found useful during my research are:

Resolving “nf_conntrack: table full, dropping packet.” flood message in dmesg Linux kernel log


http://backstage.soundcloud.com/2012/08/shoot-yourself-in-the-foot-with-iptables-and-kmod-auto-loading/

Increase IPtables hitcount limit

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

CENTOS IPtables hitcount limit.

echo “options ipt_recent ip_pkt_list_tot=60” > /etc/modprobe.d/ipt.conf
# Above value 60 is the value you want to have as max limit.

modprobe -r ipt_recent
modprobe ipt_recent

Now you are ready to make changes on your limit, and restart iptables.

Sed SRC from custom log

Mar 16 20:00:00 web1 kernel: *WEB_ATTEMPT*IN=eth1 OUT= MAC=00:208:e3:ff:08:00 SRC=111.111.111.111 DST=222.222.222.222 LEN=52 TOS=0x00 PREC=0x00 TTL=41 ID=19880 DF PROTO=TCP SPT=25499 DPT=3306 WINDOW=65453 RES=0x00 ACK URGP=0

sed ‘s/.*SRC=(.*)DST=.*/1/’ /var/log/custom | grep -v kernel | uniq -c

IPTables Logging

#Log All DB Connections
-A INPUT -p tcp -m tcp –dport 3306 -m limit –limit 5/min –limit-burst 7 -j LOG –log-prefix “*DB_ATTEMPT*”

# Logging much else clutters up the screen.
#kern.* /dev/console
kern.* /var/log/kernel

Iptables Rules

play safe with iptables:

Its always good to have these three lines on the top of iptables.

-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT

NEW – A Client requesting new connection via firewall host
ESTABLISHED – A connection that is part of already established connection
RELATED – A connection that is requesting a new request but is part of an existing connection.
INVALID – If none of the above three states can be referred or used then it is an INVAID state.

# Accept Anything from given ip

-A INPUT -s 192.168.1.1 -j ACCEPT

# Accept on certain port

-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

# Block everything from certain ip

-A INPUT -s 11.11.11.11 -j DROP

# open multiports in same line

-A INPUT -p tcp -m multiport --dports 25,53,80,110,143,443,465,587,993,995 -j ACCEPT

#Block Everything Else

-A INPUT -j REJECT --reject-with icmp-host-prohibited

# Squid; redirect.

-A PREROUTING ! -s 192.168.1.85/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING ! -s 192.168.1.85/32 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3128 


#ossec
-A INPUT -i eth0 -p tcp -m tcp --sport 1514 -j ACCEPT
-A INPUT -i eth0 -p udp -m udp --sport 1514 -j ACCEPT
-A INPUT -i eth0 -p udp -m udp --dport 1514 -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 1514 -j ACCEPT

# Reject packets from RFC1918 class networks (i.e., spoofed)
-A INPUT -i eth0 -s 10.0.0.0/8 -j DROP
-A INPUT -i eth0 -s 169.254.0.0/16 -j DROP
-A INPUT -i eth0 -s 172.16.0.0/12 -j DROP
-A INPUT -i eth0 -s 127.0.0.0/8 -j DROP
-A INPUT -s 224.0.0.0/4 -j DROP
-A INPUT -d 224.0.0.0/4 -j DROP
-A INPUT -s 240.0.0.0/5 -j DROP
-A INPUT -d 240.0.0.0/5 -j DROP
-A INPUT -s 0.0.0.0/8 -j DROP
-A INPUT -d 0.0.0.0/8 -j DROP
-A INPUT -d 239.255.255.0/24 -j DROP
-A INPUT -d 255.255.255.255 -j DROP

# Allow most ICMP packets to be received (so people can check our
# presence), but restrict the flow to avoid ping flood attacks
-A INPUT -p icmp -m icmp --icmp-type address-mask-request -j DROP
-A INPUT -p icmp -m icmp --icmp-type timestamp-request -j DROP
-A INPUT -p icmp -m icmp -m limit --limit 1/second -j ACCEPT

############## Drop Invalid Packets #################################################
-A INPUT -m state --state INVALID -j DROP
-A FORWARD -m state --state INVALID -j DROP
-A OUTPUT -m state --state INVALID -j DROP

############# Block Anything beyond 2 hit per 1/s #############
############# Second for everything

# Protect against SYN floods by rate limiting the number of new
# connections from any host to 2 per second. This does *not* do rate
# limiting overall, because then someone could easily shut us down by
# saturating the limit.
-A INPUT -m state --state NEW -p tcp -m tcp --syn -m recent --name synflood --set
-A INPUT -m state --state NEW -p tcp -m tcp --syn -m recent --name synflood --update --seconds 1 --hitcount 2 -j LOG --log-prefix "ipt: SATUR"
-A INPUT -m state --state NEW -p tcp -m tcp --syn -m recent --name synflood --update --seconds 1 --hitcount 2 -j DROP

# Above statement of synflood save from any atack however below three line is dedicated on port 80 and 443
-A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 80,443 -m recent --update --seconds 2 --hitcount 40 --name http_flood --rsource -j LOG --log-prefix "ipt: SFLOOD"
-A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 80,443 -m recent --update --seconds 2 --hitcount 40 --name http_flood --rsource -j DROP
-A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 80,443 -m recent --set --name http_flood --rsource

# Anyone who tried to portscan us is locked out for an entire day.
-A INPUT -m recent --name portscan --rcheck --seconds 86400 -j LOG --log-prefix "ipt: Portscan"
-A INPUT -m recent --name portscan --rcheck --seconds 86400 -j DROP

-A FORWARD -m recent --name portscan --rcheck --seconds 86400 -j LOG --log-prefix "ipt: Portscan"
-A FORWARD -m recent --name portscan --rcheck --seconds 86400 -j DROP

############## Incomming packet should always have SYN Packet########################
-A INPUT -p tcp ! --syn -m state --state NEW -j DROP

################################## DROP INVALID SYN PACKETS##########################
-A INPUT -p tcp --tcp-flags ALL ACK,RST,SYN,FIN -j LOG --log-prefix "ipt: ALLI"
-A INPUT -p tcp --tcp-flags ALL ACK,RST,SYN,FIN -j DROP

# Drop bogus TCP packets
-A INPUT -p tcp -m tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
-A INPUT -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP

# Lets see if this affects legitimate traffic on RST per 2 sec
# Drop excessive RST packets to avoid SMURF attacks, by given the
# next real data packet in the sequence a better chance to arrive first.
-A INPUT -p tcp -m tcp --tcp-flags RST RST -m limit --limit 2/second --limit-burst 2 -j ACCEPT

################ Incoming malformed XMAS packets drop them:##########################
-A INPUT -p tcp --tcp-flags ALL ALL -j LOG --log-prefix "ipt: XMAS"
-A INPUT -p tcp --tcp-flags ALL ALL -j DROP

################## Drop all NULL packets############################################
################## Incoming malformed NULL packets:#################################
-A INPUT -p tcp --tcp-flags ALL NONE -j LOG --log-prefix "ipt: NULL"
-A INPUT -p tcp --tcp-flags ALL NONE -j DROP

############ Block fragments ####################################
-A INPUT -i eth0 -f -j LOG --log-prefix "ipt: Frags"
-A INPUT -i eth0 -f -j DROP

-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j LOG --log-prefix "ipt: NONE"
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP

secure asterisk server with iptables

Secure Asterisk Server

iptables -A INPUT -p udp -m udp --dport 10000:20000 -j ACCEPT
# This has to be port open on rtp.conf

iptables -A INPUT -p udp -m udp --dport 5060 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 5038 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT
iptables -A INPUT -p tcp --dport 1720 -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

And reject all

Securing Asterisk Server (Simple tips)

Securing Asterisk Server (Simple tips)

Change Admin: User: Done

Use Monitoring such as Ossec for any suspecious alerts.

Change root password if its simple

vi /var/www/html/recordings/includes/main.conf
Console= disabled

Extension passwords to be complicated (Auto generated)

Iptables

-A INPUT -i lo -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

#Open for Web UI,SSH and SSL
-A INPUT -p tcp -m multiport --dport 80,443,22 -j ACCEPT

# IP to allow SIP Port
-A INPUT -p udp -m udp -s x.x.x.x/32 --dport 5060 -j ACCEPT

# DDI Providers IP All range
-A INPUT -p udp -m udp -s xx.xx.Xxx.x/25 --dport 5060 -j ACCEPT

# RTP PORT as mentioned in rtp conf file
-A INPUT -p udp -m udp --dport 10000:20000 -j ACCEPT


#Block Everything Else
-A INPUT -j REJECT --reject-with icmp-host-prohibited

From Freepbx web or conf

For each Extension:

deny = 0.0.0.0/0.0.0.0 (All)
Permit = XX.XX.XX.XX/255.255.255.255 (Only My IP)

ICMP Parameters

vi /etc/sysconfig/iptables

inbound icmp block

-A INPUT -p icmp --icmp-type echo-request -j DROP

outbout icmp block

-A OUTPUT -p icmp --icmp-type echo-request -j DROP

It can be blocked with Numbers

iptables -A OUTPUT -p icmp --icmp-type 8 -j DROP

Here you can find ICMP Numbers
http://www.iana.org/assignments/icmp-parameters/icmp-parameters.xml

track SSH traffic (bandwidth usage)

track SSH traffic (bandwidth usage)

I would use iptables owner module (perhaps together with other quota/reporting modules).
iptables -A OUTPUT -p tcp –dport 22 -m owner –uid someuser -j ACCEPT

iptables -vL

IP Tables rules to allow only those from certain Ip’s

IP Tables rules to allow only those from certain Ip’s

-A INPUT -s xxx.xxx.xxx.xxx -p tcp -m tcp –dport 3306 -j ACCEPT # Allow from
-A INPUT -s xxx.xxx.xxx.xxx/27 -p tcp -m tcp –dport 3306 -j ACCEPT
-A INPUT -p tcp -m tcp –dport 3306 -j DROP

Powered by WordPress & Theme by Anders Norén