Wedebugyou

We all heard about DDoS and we all fear them. We all hope to never be a victim of hackers or be ask for a ransom in exchange to not be DDoS. The day it’s happening to you, you realize it’s too late. You call your ISP and they tell you it will costs 3000$ to filter the DDoS. The attacker ask for 1000$ in exchange to not DDoS you. You don’t know what to do and then you start to panic.

We will explain you how to prevent a DDoS in part1 of this blog. This means the steps to do before a hacker contacts you or if you prefer: preparation. The second part will contain the steps to mitigate a DDoS or what to do while you are under attack.

The steps to prevent a DDoS can be resume in this:

1 – Increase the number of concurrent connections
2 – Apply an antispoof ruleset
3 – Enable SYNproxy
4 – Prepare a black list
5 – Configure Rate-limit

[gard]

A little story

But before we go in the details have you red The Pillars of the Earth from Ken Follett? In this novel a little village call Kingsbridge build a cathedral. The rival village is jealous and decide to raid them. They killed half of the villagers and burned everything. Many years later they restarted to build the cathedral. Again, the story repeat but, this time a young knight heard that they will raid. He returned to Kingsbridge and advise the villagers that they are coming to raid again. They have 2 or 3 days of preparation. They decided to build a giant wall all around the village. This way the villagers will be protected. It worked!

:)

Let’s build a wall but let’s take our time.

Preparing for a DDoS involve many hours and sometimes many days of preparation. The bigger your online infrastructure is, the more complex it is to protect.

Warnings

– Don’t blindly apply any modifications proposed here in your live environment without proper testing.
– Watch out with the logging. If you enable too much logging you could DDoS yourself. Log only if it’s necessary.
You’ve been warned.

Hardware and software

For the demonstration we will use pfSense 2.1. We assume you will use decent hardware. To protect DDoS up to several 100K connections, you will need at least quad-core 3.0 GHz with 4 GB RAM. The older generation of server hardware like dell 2850 and hp G5 and under can transfer less traffic than the newer generation of hardware. If you have the newer generation you can probably increase a bit the metrics proposed.
The concept that will be explain here also adapt to linux iptables or any kind of proprietary hardware provider like cisco, f5, juniper, checkpoint, etc.

1 – Increase the number of concurrent connections

When you are under DDoS the number of concurrent connections will be abnormally high. You still want to handle them all because your legitimate client will also need a connection to reach you. It’s not clear as how much RAM is required per connection. A good rule of thumb is 2 GB of RAM per 1 000 000 connections.

In pfsense, click on system menu then advanced. Then click on the Firewall/NAT tab.

Change the values of Firewall Maximum States and Firewall Maximum Table Entries.

If you need several millions concurrent connections, then it’s preferable to load-balance the traffic on several different firewalls. This technique will not be describe in this post.

2 – Apply an antispoof ruleset

In the mid 90’s every linux/unix howto available on internet included an antispoof ruleset. It seems that today many enterprises forget about this basic technique. Remember this one, it’s very important. If you want to know more about antispoof you can check this page.

To enable antispoof in pfSense click on the interfaces menu and select WAN. Enable the 2 options Block private networks and Block bogon networks.

* If you are testing or your WAN for whatever reason is in a RFC 1918 (10/8, 172.16/12, 192.168/16) network, don’t enable this feature as you will paint yourself into a corner.

3 – Enable SYNproxy

When you will be under DDoS many kind of attacks will arrive at the same time. One of them is call the SYN flood and it’s when the attackers try to open as many TCP connections as possible. They often open them in strange state like “half open”. The web, mail, database servers are not good at handling many connections in weird states. It’s important that they never receive this kind of junk traffic. The way to effectively protect your online servers is to use SYN proxy.

Let’s protect our web servers.
In pfSense click on Firewall then Rules.
Modify your firewall rules that allow http traffic to your web servers. Scroll down until you reach a section with State Type. Change the value from default keep state to synproxy state.

You will activate this rule only under attack.
It’s not good to keep it permanently on. The trick is to create a SYNproxy rule above a normal keep state rule. You should keep the SYNproxy rule disable and you just enable it under attack. You know that you have a SYNproxy rule when you see a small purple a beside the rule.

Add as many SYNproxy rules as you need. You should protect what is critical for your business.

[GARD]

4 – Prepare a black list

When you will be under a DDoS attack, you should notice that some countries generate more traffic than others. If you know that your clients are base in the US, or in some countries of Europe, you should prepare a black list that will block all the other countries. This black list will be apply under the DDoS and will help to block a huge quantity of bad traffic.

4.1 – Install pfBlocker.
In pfSense click on System then packages. Click on the available packages tab. In the list find pfBlocker and click on the little + to install it. Once install click on Firewall then pfBlocker. Enable the options like this and make sure to not Enable Logging.

We block on the WAN interface to not reply to those bots or zombies. Also it’s important to reject the outgoing traffic from the online servers in direction to those hackers. Maybe they found a hole in your system and they will want to download more hacking tools from these black listed countries from the compromise server. When you reject the firewall will send an icmp port unreachable to your server. This will reset the connection and the application will not time out. It helps your servers.

Recommendations
– We recommend to not allow your online servers to initiate connection toward internet at all time. We know it’s not easy to enforce but, if you can this rule will improve a lot the security of your network.
– We recommend to not run these lists permanently.
– If you want to run a basic black list of the most annoying country to your environment it’s ok. Just make sure to not select all and run without testing.
– You should prepare a basic black list and when you will be under DDoS, you can update it. Then over time you will build a custom black list for your enterprise.

So you ctrl+click on some countries in the several continents of pfBlocker and you configured the options for your needs? Great! We are now ready to prepare the last task. It’s also the most complicated and difficult one.

5 – Configure Rate-limit

Like the name say, in this task we will define some predefine limits. When a bot or a zombie attack one of our ressource and reach that limit, the firewall will instantly drop all connections from that ip address. The problem with this tool is we can also block legitimate clients if we don’t define good tresholds. That’s why it’s important to prepare and test before you are under DDoS.

5.1 – How do we test and configure rate-limit?
If you prepare yourself good, this will be the ultimate shield against DDoS and this is where you will spend the most of your energy and time tweaking. If you have 15 ip addresses this step is much more easier than if you own few thousands ip addresses. Now we assume that you understand your online environment. You should have an idea of how much connections per services you have. If you don’t you should start a side project to gather these informations. Tools like Netflow are very usefull for that. Also for smaller project ntop do a very good job.

In pfSense, click on Firewall then Rules. In your ruleset, edit the rule you want to rate-limit. Scroll down to the bottom and click on Advanced Options.

Here you define your defense strategy.
In example if your online users use in average 10 connections per visit. They use your online services for about 10 minutes then they come much later. You could define a rule of 20/180 Maximum new connections / per second(s). This is more than enough for all your users. The attackers will not be so gentle with you when they will attack. They will realize too late that all their bots/zombies are now block by our rate-limit line of defense.

5.2 – Find your own values
There is no magic formula that we can write here and that will adapt for everybody. You will need to find your own metrics and values. A trick to identify good values is to start with very high values and log the rules if somebody hits it. Try to identify if it was a distributed scan, some kind of attack or really a legitimate client. Then if your value is good lower it a little bit and monitor again. Find your own values and document them.

Recommendations
– Edit the SYNproxy firewall rule, that we created earlier, and apply aggressive rate-limit values. This rule will be turn on only under attack. Make sure to not log this rule.
– Download Loic and DDoS yourself. If your rate-limit block you then it will also block them.
– Edit the equivalent rule but with keep state and configure a loose rate-limit. Make sure to log it and that it sends you an email if it’s reach. Then you will know that a DDoS is in preparation or maybe already started.
– Test and document all your online critical services: dns, email, web applications, vpn, etc
– Retest your rate-limit few times a year or at least once a year.

[gard]

How to mitigate a DDoS?

If you prepare yourself like we describe in this blog you will have a long line in front of your attackers. In fact, they will be surprise and they will not understand what’s happening. They will then add more beasts and try more aggressive patterns. The battle will change and the tricks to apply during a DDoS will be explain in another blog. We will also answer the question of how to block the traffic before it’s sent to you in a way that it doesn’t saturate your internet line. Before we explain what to do during an attack it’s important to have a strong defense strategy.

Conclusion

We don’t pretend that these preparations will solve and block all DDoS. Also there are more things to do to be fully DDoS protected. At least you will have strong walls around your critical online environment and be ready for an attack. This way you will not be an easy prey.

Tell us what you think? Are you prepare to face a DDoS? Were you already a victim of DDoS? What happenned? Share with us what do you think about DDoS.