GitHub hit with record 1.35-Tbps denial of service attack, more attacks expected

GitHub suffered and survived a record 1.35-terabit-per-second denial of service attack on Wednesday, an unprecedented deluge of traffic that’s spotlighting just how powerful “amplification attacks” can be — and a new attack vector experts predict is about to become a lot more common.

The top comment on the Hacker News discussion says it all: “Wow, 1.35Tbps? That’s a lot for a DoS attack, right?”

It’s still early in 2018, but that could be the understatement of the year so far. Wednesday’s attack counts as the most powerful denial of service barrage against a single site in history. It’s significantly larger than the size of the 2016 Mirai botnet attacks that brought down a host of the internet’s biggest websites through an attack on Dyn that rippled out to other sites dependent on the company’s infrastructure and DNS services.

GitHub — a popular software development platform — went down a number of times during this week’s attack until traffic was moved to Akamai, a cloud computing company that provided protection from the flood.

Here’s how the attack spiked a little after noon Eastern time on Wednesday:

(GitHub)

The tactic is known as an “amplification attack.”

It works like this: An attacker spoofs their IP address to look like the victim’s IP address. They send a forged request to a vulnerable memcached server — databases that are meant to speed up networks but lately are abused because many are inadvertently exposed to the public internet. That’s a problem receiving more attention in the aftermath of this attack.

Unaware that it’s being pinged by an attacker as opposed to a normal machine, the server responds to the victim’s IP address. The response packets are much larger than the original queries, so an attacker with 1-gigbit-per-second capacity can ultimately launch a 100-Gbps attack, hence the name “amplification attack.”

The Mirai botnet required around 100,000 infected machines to reach just over 1 terabit per second while this attack appears to have less than a tenth of that number while launching a larger attack.

When an attacker performs this against thousands of memcached servers, you get a “memcrash“ as huge amounts of data are sent against the unsuspecting victim by the gullible servers.

It’s a new attack vector first observed earlier this week, just about one day before the attack against GitHub. The method was first discovered by researchers at the Chinese cybersecurity firm Qihoo 360 in 2017. They predicted seeing denial of service attacks measuring up to 2 terabits per second.

“The attack originated from over a thousand different autonomous systems (ASNs) across tens of thousands of unique endpoints,” GitHub’s Sam Kottler wrote in a Thursday morning post-mortem. “It was an amplification attack using the memcached-based approach described above that peaked at 1.35Tbps via 126.9 million packets per second.”

There are no huge collections of enslaved computers to form a botnet, just a series of forged requests that can result in huge denial of service attacks. Here’s a Cloudflare explainer:

(Cloudflare)

GitHub wasn’t alone in suffering attacks, according to Akamai, which wrote that “many other organizations have experienced similar reflection attacks since Monday.”

Mitigating these attacks is doable by blocking connections on port 11211, the default port used by memcached, exactly the way Akamai’s Prolexic platform was able to mitigate these attack.

Both Cloudflare and Akamai predict “many more, potentially larger attacks” in the near future. Both companies are soliciting help to find out who sent the massive attack as well as to fix the vulnerable protocols and IP spoofing that make the attack possible.