MENU
ides-2

Beware the Ides of March: Subsea Cable Cut Trend...

dyn_on_yellow_2

Renesys, Dyn Join Forces

April 3, 2014 Comments (22) Views: 3034 Business, Internet, Security, South Asia, Uncategorized

Indonesia Hijacks the World

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInShare on Reddit

indonesia2

photo by null0 on Flickr | CC

Yesterday, Indosat, one of Indonesia’s largest telecommunications providers, leaked large portions of the global routing table multiple times over a two-hour period. This means that, in effect, Indosat claimed that it “owned” many of the world’s networks. Once someone makes such an assertion, typically via an honest mistake in their routing policy, the only question remaining is how much of the world ends up believing them and hence, what will be the scale of the damage they inflict? Events of this nature, while relatively rare, are certainly not unheard of and can have geopolitical implications, such as when China was involved in a similar incident in 2010.

Keep in mind that this is how the Internet is designed to work, namely, on the honor system. Like Twitter and Facebook, where you can claim to be anyone you want, Internet routing allows you to lay claim to any network you want. There is no authentication or validation. None. But unlike Twitter and Facebook, such false claims propagate through the world in a matter of seconds and decisions, good or bad, are made algorithmically by routers, not humans. This means that innocent errors can have immediate global impacts. In this incident, the impacts were most pronounced on Akamai, one of the world’s largest content delivery networks, which was a very bad thing. Akamai hosts thousands of networks for their customers, including turbotax.com, healthcare.gov, paypal.com and many other high-profile sites.

The trouble with Indosat began at 18:25 UTC yesterday when they leaked over 320,000 routes. Since a full routing table currently contains nearly 500,000 routes, this means that Indosat laid claim to roughly two-thirds of the Internet!

While many of these routes didn’t travel very far from Indonesia and hence, would not have had much of an impact on Internet traffic, a few hundred were widely accepted, and a large fraction of these belonged to Akamai.

Besides disrupting Akamai themselves, this routing leak completely took out Indosat in what amounted to a self-inflicted DDoS attack. Our global latency measurements into this ISP via all of their upstream providers all but stopped during this time period and remained impaired even after the bogus routing announcements were withdrawn.

Traces-to-Indosat Indosat-DDOS

Surprisingly, for some Akamai prefixes (networks), the Indosat hijack was essentially complete, with most of the world choosing Indonesia as the best place to send this particular Akamai traffic.

indosat_leak_akamai2

For others who were impacted, the hijacking was partial, with some of the world selecting Indosat and others selecting the rightful owners. For example, Chevron in London saw about half our routing sources choosing Indonesia over the UK during much of this 2-hour disruption.

indosat_leak_chevron

 

We can assess the probable operational impact on each affected network by examining this split between our peers who selected Indosat (and therefore would have supported traffic misdirection) and those who stuck with the real owner’s routes. Several hundred thousand networks were affected to some degree, but 99.7% of these were minimally affected, with less than 5% of our peerset convinced to take the alternative Indosat origin. We can divide the remaining 0.3% into three tiers:

  • Low Impact (0.2% of affected networks): potential traffic redirection affected more than 5%, but not more than 25%, of our peers. Examples:
    • PNC Bank NA, US (12.48.101.0/24)
    • CreditWest Bank, Turkey (37.235.79.0/24)
    • Nova Banka Banjaluka (5.133.0.0/21)
    • Starcard Banka Kartlari Merkesi, Turkey (37.235.72.0/24)
  • Medium Impact (0.06% of affected networks): potential traffic redirection affected more than 25%, but not more than 50%, of our peers. Examples:
    • Halliburton, Norway (34.253.128.0/22)
    • Fedex, US (12.31.21.0/24)
    • Capstone Financial Advisors, US (12.48.63.0/24)
    • Complex Financial Systems Ltd, Russia (31.40.76.0/22)
    • Bank-Inform, Ukraine (37.46.224.0/20)
    • New People’s Bank, US (12.43.216.0/24)
    • Citrix Online, India (202.173.29.0/24, see below)
  • High Impact (0.03% of affected networks): more than 50% of our peers routed traffic via Indosat instead of the true owner. Examples (in addition to Akamai-hosted blocks):
    • Stan Telecom, Afghanistan (27.116.59.0/24)
    • Chevron Corporation, UK (146.23.210.0/24)
    • City of Santa Monica, California (66.198.19.0/24, see below)
indosat_leak_santa_monica indosat_leak_citirx

Conclusions

In the absence of a single world government (for strict authentication) and much greater controls over Internet routing (for strict validation), there is currently no way to completely prevent these types of incidents. In the same way that anyone can set up a fake Facebook account with your name on it, so too can any router in the world claim to be the best way to reach your network. At the very least, enterprises need to be monitoring and managing their own Internet assets, as not all hijacks are necessarily innocent or short-lived.

Enterprises also need to carefully police their own routing policies and understand how the world reaches them. The reason why Chevron was impacted globally was largely of their own making: normally, they heavily prepend their BGP announcements through British Telecom, one of their providers. That is, the AS paths to 146.23.208.0/21 tend to look like … 2856 7862 7862 7862 7862 7862. By this mechanism, Chevron has artificially lengthened its AS path, thereby de-prioritizing the selection of this route. Unfortunately, this approach has also left them open to hijacking, since BGP route selection uses AS path length in its decision-making process. When Indosat starting leaking routes, the heavily prepended (and correct) Chevron routes were some of the first to be misdirected to Indonesia, as AS paths via Indosat were often shorter.

We saw this same behavior back in April of 2010 during China’s routing leak. Some of the worst impacted routes in that incident were from Charlottesville, Virginia. Not because China was targeting this college town of 43,000 in central Virginia, but because those routes were heavily prepended at all times, all but guaranteeing any errant routes from anywhere in the world would be preferred.

In short: route leak events like this one, which happen at least once a year, are a good reminder that BGP routing is fragile and error-prone. There are no easy fixes. That means that every enterprise on the Internet should be monitoring the advertisements of their networks, keep published ASPATHs compact and free from unnecessary prepending, and be prepared to temporarily advertise one or more more-specific routes, if possible, to win back control of inbound traffic. Don’t be part of the 0.03% who suffer serious impacts from large accidental route leaks.

22 Responses to Indonesia Hijacks the World

  1. frnkblk says:

    This posting doesn’t mention RPKI….

  2. […] ma możliwość dokładnej obserwacji globalnego ruchu internetowego i na tej podstawie przedstawiła kilka ciekawostek związanych z tym […]

  3. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  4. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  5. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  6. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  7. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  8. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  9. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  10. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  11. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  12. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  13. […] made it seem Indosat managed some 320,000 of 500,000 networks on the Web for roughly two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  14. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  15. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  16. […] done it seem Indosat tranquil some 320,000 of 500,000 networks on a Internet for about dual hours, wrote Earl Zmijewski, a clamp boss and ubiquitous manager for Renesys, on a association […]

  17. […] Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  18. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  19. […] it appear Indosat controlled some 320,000 of 500,000 networks on the Internet for about two hours, wrote Earl Zmijewski, a vice president and general manager for Renesys, on a company […]

  20. […] Indonesia Hijacks the World – Renesys That was an incident that happened earlier this month where 2/3 of Internet traffic was redirected to Indosat's networks. I've been asked how could I prevent such a mess, but I have no idea. Quote   […]

  21. Has route filtering (manual or automatic based on things like RADB) been abandoned by most providers?  When I was running an ISP ~10 years ago some transit providers required an LOA before they’d update their route filters to allow me to advertize a new customer’s address block.

  22. […] ricercatore ci fa poi notare che i problemi ci sono già stati: nell’aprile 2014 furono diffuse molte tablle di routing e Akamai – il più grande network per la distribuzione di […]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>