February 17, 2009
This is the second in a series of post summarizing some of the information I presented in the recent SANS roundtable webcast that I participated in with fellow SANS instructors Ed Skoudis and Mike Poor. Hopefully this written summary will be of use to those of you who lack the time to sit down and listen to the webcast in its entirety.
During the webcast we were discussing the recent distributed denial-of-service (DDoS) attacks against the Immunity, Metasploit, Milw0rm, and Packet Storm web sites. H.D. Moore has been posting brief updates about the attacks on his blog, and there are some interesting lessons to be extracted from his commentary. It’s also good reading for H.D.’s scathing commentary on the quality of the attack.
To summarize, the attacks began around 9pm CST on Friday 2/6. The attack was a SYN-only connection attack on H.D.’s web servers on 80/tcp. H.D. observed peak traffic loads of approximately 80,000 connections per second.
When faced with a DDoS like this, one of the first questions is are the source IPs on the traffic real or spoofed? H.D. was able to observe his DNS server-related logs and see that about 95% of the source addresses were resolving metasploit.com periodically. This tends to indicate that the attacker was not bothering to spoof source IP addresses in the traffic.
It also led to the first corrective action to thwart the DDoS: H.D. simply changed the A record for metasploit.com to 127.0.0.1. The downside to this action is that now legitimate users couldn’t find metasploit.com, and in particular it broke the standard update mechanism for the Metasploit Framework. However, since the attack was only targeting the main metasploit.com web site, H.D. was able to move his other services (blog.metasploit.com, etc) to an alternate IP address and keep operating for the most part. He was also able to use various “Social Media” outlets (like Twitter, his blog, etc) to let people know about the changes and help them find the relocated services (I can hear the Web 2.0 press organs hooting now, “Twitter Foils DDoS Attack!”– *blech*).
Actually, H.D. is now in sort of an odd situation. The attackers have hit him with a fire hose of data, but effectively left him controlling the direction of the nozzle. If H.D. were less ethical than he is, he simply could have changed his A record to point at a routable IP address and used the attack to DDoS somebody else. It’s unlikely that the folks at the re-targeted site would even have realized why this traffic was suddenly being directed at them, since the attacker wasn’t bothering to send actual HTTP transactions with information about the original target site.
In any event, the attack finally abated at about 9pm CST on Sunday, 2/8. However, it restarted again Monday morning (read H.D.’s comment regarding the timing of the attack– priceless). This time, the attacker appears to be using the hard-coded IP addresses of both the original metasploit.com web site and the alternate IP that H.D. had relocated to during the initial attack. So changing the A record would no longer be effective at stopping the attack.
However, the attackers were still only targeting port 80/tcp. All H.D. had to do is shut down the web servers on 80/tcp and restart them on 8000/tcp (again notifying his normal users via Twitter, etc). H.D. notes that his SSL traffic on 443/tcp was completely unaffected by the DDoS, which again was a pretty poor showing by the attackers.
And so the attack continues. By Tuesday, the SYN traffic has reached 15Mbps/sec, and H.D.’s ISP is starting to get worried. However, by this point the DDoS sources are apparently back to using DNS resolution because H.D. notes he’s pointing the metasploit.com A record back to 127.0.0.1 and moving his services to metasploit.org. Of course, at some point the attackers are going to target H.D. at his new domain. Maybe H.D. should consider deploying the Metasploit web presence into a fast-flux network to continue evading the DDoS weasels?
In any event, I think the countermeasures that H.D. took against the DDoS (changing the A record, changing IPs, changing ports) are all good stop-gap approaches to consider if you’re ever targeted by a DDoS attack like this. I also commend H.D. for his use of “social media” as an out-of-band communications mechanism with his regular constituents. And finally, the transparency of blogging about the attack to inform the rest of the community is certainly helpful to others. Good show, H.D.!